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ABSTRACT 


This  research  analyzes  two  approaches  to  the  economic  evaluation  of  an  aircraft 
modernization  program.  The  Analytic  Science  Corporation  (TASC)  method  is  compared 
with  the  Logical  Decision  for  Window  (LDW)  methodology.  TASCFORM-AIR  model 
is  a  method  to  quantitatively  measure  military  force  modernization.  Logical  Decisions  for 
Windows  software  and  methodology  is  based  on  Multiattribute  Utility  Theory.  It  also 
helps  to  evaluate  decisions  quantitatively. 

The  research  includes  analysis  of  the  reasons,  constraints  and  tendencies  in  the 
modem  aircraft  modernization  process.  Weapon  modernization  is  usually  driven  by 
several  objectives,  all  of  them  in  one  way  or  an  another  are  pertinent  to  resource 
allocation.  Reliable  analytical  tools  are  important  to  make  good  decision.  Cost- 
effectiveness  and  cost  utility  approaches  are  evaluated. 

Comparison  of  both  methodologies  is  based  on  the  MiG-29  modernization 
situational  model.  TASCFORM-AIR  Model  provides  static  indicators  of  military  force 
potential.  This  can  be  viewed  as  measures  of  effectiveness.  The  LDW  program  computes 
the  alternatives’  utility  by  combining  its  measure  levels  based  on  the  analyst’s 
preferences.  The  results  produced  in  both  cases  are  useful  in  several  ways.  They  are 
indicators,  however  indicators  rather  than  "answers"  to  the  decision  making  problem 
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I.  INTRODUCTION 


A.  BACKGROUND 

There  is  no  need  to  be  an  expert  in  aviation  to  notice  a  significant  drop  in  the 
production  and  design  of  new  combat  aircraft  at  the  present  time  in  all  leading  countries. 
Among  the  main  reasons  for  this,  one  can  mention  the  end  of  the  Cold  War  and  the 
collapse  of  the  Soviet  Union;  others  will  emphasize  economic  difficulties  and  the  rapidly 
increasing  costs  for  new  aircraft  design  and  production.  The  countries  making  new 
weapon  platforms  produce  quantities  of  aircraft  much  less  than  the  levels  of  the  70s  and 
80s. 

The  days  when  the  United  States  and  the  Soviet  Union  produced  a  new  class  of 
combat  aircraft  every  ten  years  and  a  new  transport  every  fifteen  years  are  gone. 
Procurement  of  new  military  aircraft  now  is  more  a  political  and  economic  issue  than 
purely  military.  In  addition,  the  life  cycle  cost  of  the  whole  system  plays  as  important  a 
role  in  source  selection  as  combat  characteristics. 

Nevertheless  all  military  doctrines  and  recent  combat  experience  show  the 
increasing  role  of  aviation  in  the  structure  of  military  forces.  For  the  military  economist, 
it  means  that  to  keep  old  aircraft  safe  and  able  to  perform  modem  operational  tasks 
require  some  additional  money.  In  those  conditions  many  countries  have  chosen  the 
upgrade  and  modernization  path.  Many  studies  show  that  most  jet  combat  aircraft  of  the 
so-called  “third  generation”  have  great  potential  for  modernization  and  upgrade  using 
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modem  technology.  Even  aircraft  veterans,  such  as  the  MiG-21  and  the  F-5,  will  get  new 
life,  and  the  description  of  the  modernization  programs  for  MiG-29  and  F-15  have 
appeared  in  all  specialized  magazines. 

In  many  cases,  industry,  trying  to  find  new  markets,  proposes  its  own  programs  to 
customers.  But  generally,  aircraft  upgrade  programs  tend  to  be  customer  driven,  with 
industry  responding  with  a  solution  tailored  to  specific  customer  requirements.  On  the 
other  hand,  a  number  of  pitfalls  exist  on  this  path.  Two  basic  conditions  have  vital 
importance  for  any  aircraft  upgrade  plans.  First,  the  basic  airframe  must  have  an 
acceptable  future  service  life  and,  second,  ensuring  an  adequate  holding  of  basic  spare 
parts  is  important. 

The  aircraft  built  before  1970  had  service  lives  of  15-20  years.  Later,  the 
electronic  component  of  the  aircraft  became  increasingly  important,  and  in  recent  models 
designers  have  pre-planned  the  mid-life  upgrade.  Today,  the  cost  of  the  frame  and  the 
engines  are  probably  less  than  50%  of  the  total  aircraft  cost,  and  aircraft  such  as 
“Eurofighter”  have  built-in  upgradeability  from  the  start  that  allows  the  full  potential  of 
the  upgrading  to  be  realized. 

Clearly,  upgrading  and  modernizing  are  only  a  partial  solution.  Future  needs  will 
demand  replacing  old  machines  with  new  ones.  Even  the  best  modernization  program  is 
not  a  panacea  for  new  procurement,  but  modernization  can  buy  the  user  time  to  consider 
the  alternatives.  Modernization  can  be  a  direct  replacement  by  buying  a  new  generation 
of  the  same  class  of  aircraft,  or  it  can  be  indirect,  by  giving  an  aircraft  a  new  role. 
Modernization  “  can  reduce  cost-of-ownership  for  the  later  part  of  an  extended  service 
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life,  and  it  can  offer  interim  capability  until  the  next  generation  comes  along  or  an 
improvement  is  made  in  an  existing  role.  The  decision  to  upgrade  lies  with  the  user 
alone.”  (Gething  p.  2) 

The  never-ending  researches  of  designers  and  producers  of  the  aircraft,  engines, 
equipment  and  armament  increase  the  number  of  alternatives  available.  What 
combination  to  chose?  Which  characteristics  are  more  important?  Having  constrained 
budgets,  military  planners  often  face  complicated  decision  problems  with  great 
uncertainties  and  multiple  inputs  and  outputs.  Good  intuition  and  rich  experience  are  not 
sufficient  to  reach  the  best  solution.  Modem  economic  and  military  sciences  offer  a  great 
number  of  analytical  instruments  for  decision  making  and  support.  Which  one  best  fits  a 
modernization  decision? 

In  my  thesis  I  will  evaluate  widely  used  scientific  analytical  methodologies  as  to 
their  practical  use  by  military  managers  for  modernization  decision-making. 

B.  PURPOSE 

By  considering  the  vital  role  of  research  tools  for  effective  allocation  of  budgetary 
recourses,  I  will  evaluate  selected  methods,  approaches,  and  real  programs  for  decision¬ 
making  in  aircraft  upgrade  and  modernization. 

The  main  purpose  of  this  thesis  is  to  estimate  data,  examine  assumptions  and 
evaluate  the  results  of  different  methods  and  to  make  recommendations  for  their  practical 
use  within  the  Armament  Department  of  the  Ukrainian  Ministry  of  Defense.  The  case  of 
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the  MiG-29  modernization  will  be  used  to  evaluate  research  tools  and  to  illustrate 


recommended  methodology. 

C.  RESEARCH  QUESTIONS 


The  research  questions  due  to  the  nature  of  the  problem,  and  because  of  the  use  of 
situational  modeling  for  methodology  evaluation  are  divided  into  two  groups. 

Group  I.  Pre-modeling  questions: 

•  What  are  the  main  reasons  for  combat  aircraft  modernization  and  what  are 
the  objectives  and  the  constraints  on  modernization? 

•  What  generalization  and  assumptions  can  be  made  in  modernization 
modeling? 

•  How  can  cost-effectiveness  and  cost-utility  analysis  be  used  for 
modernization  decision-making? 

•  What  specialized  and  universal  decision  support  programs  and  software 
exist? 

Group  II.  Post-modeling  questions: 

•  What  criteria  can  be  used  for  methodology  selection? 

•  What  are  the  key  dimensions  of  the  methodology  selection  problem? 

•  What  are  the  main  measurement-scale  selection  and  construction  issues 
relevant  to  given  multi-attribute-utility  analysis? 

•  What  are  the  advantages  and  disadvantages  of  assessed  methodologies? 
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II.  AIRCRAFT  MODERNIZATION  DECISION 


The  improvement  or  replacement  of  military  weapons  system  is  generally 
referred  to  as  weapon  modernization.  The  ostensible  purpose  of  such 
modernization  is  to  correct  shortcomings  in  the  original  weapons,  or 
respond  to  either  changes  in  the  weapons  purpose  or  to  changes  in  enemy 
weapons.  (Sorenson,  p.  23) 

There  are  three  ways  to  modernize  existing  weapons.  Modem  aircraft  as  a  system 
can  be  improved: 

•  as  a  platform  (speed,  maneuverability,  range). 

•  as  a  part  of  strike  complex  (command  control,  communication  and 
navigation). 

•  as  a  weapon  (power,  accuracy,  effectiveness). 

Modernization  may  change  just  a  single  element  of  a  weapon  system  or  it  can  be  a 
part  of  state  weapon  program.  For  example,  the  latest  modification  of  the  F-16  after  a 
midlife  upgrade  is  believed  to  be  two  or  three  times  more  effective  that  its  predecessor. 
(Sweetman,  p.  26) 

Technologically  speaking,  modernization  can  be  as  evolutionary  as  the  case  of  the 
MiG-21  that  was  gradually  transformed  many  times  during  its  45-year  history,  or  as 
revolutionary  as  when  one  technology  is  completely  replaced  by  another.  For  example, 
the  transformation  from  piston  to  jet  engines  in  aviation  was  clearly  revolutionary. 

Technology  can  influence  not  only  changes  in  weapons,  but  can  also  transform 
war  itself.  In  this  Chapter,  I  will  evaluate  modernization  as  a  subject  of  policy  that  can 
impact  the  defense  resource  allocation. 
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A.  REASONS  AND  OBJECTIVES 


Weapon  modernization  is  usually  driven  by  several  objectives,  all  of  them  in  one 
or  other  way  are  pertinent  to  resource  allocation.  Modernization  can  result  from  national 
security  requirements  or  particular  service  demands  based  on  strategy  development.  In  a 
political  context,  it  can  arise  from  regional  or  group  interests.  Analyzing  military 
aviation  development  we  can  define  how  the  following  typologies  explain  weapon 
modernization: 

•  Modernization  due  to  strategic  necessity.  National  security  defines  threats 
and  then  strategy  produces  weapon  performance  demands; 

•  Weapon  modernization  based  on  demand  to  match  a  new  military 
capability  of  a  rival  nation; 

•  Modernization  based  on  interest  of  powerful  domestic  groups  that 
benefited  from  military  founding; 

•  Weapon  modernization  as  a  means  to  maintain  a  defense  industrial  base; 

•  Modernization  opportunities  based  on  scientific  and  technological 
innovations; 

•  Modernization  as  the  result  of  shifted  control  over  the  military  acquisition 
process. 

We  can  separate  these  reasons  on  two  levels:  strategic  (the  first  two)  and 
structural  (the  others).  The  first  group  is  connected  to  the  international  environment  as 
the  stimulus  for  weapon  development;  the  second  group  focuses  on  domestic  policies. 

To  better  understand  the  modernization  decision  making  process,  I’ll  evaluate 
separately  the  influence  of  state  strategy  and  military  competition,  the  role  of  military 
doctrine  and  the  place  of  modernization  within  the  military  planning.  To  make  this 
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analysis  more  relevant  to  methodology  selection,  I  will  evaluate  the  technical  level  of 
modernization  caused  by  flight  safety,  effectiveness  and  obsolescence.  Even  though 
technical  problems  are  partly  included  in  strategic  and  structural  levels,  evaluating  them 
separately  is  reasonable  in  the  context  of  this  thesis. 

1.  Strategy  and  Doctrine 

Strategy  connotes  the  linkage  between  national  political  objectives  through  the 
threat  or  use  of  the  military  forces.  (Sorenson,  p.  26)  For  example,  for  the  United  States, 
the  Cold  War  strategy  was  a  combination  of  deterrence  and  containment.  Both  were 
based  on  the  expectation  of  expanding  Soviet  power  and  on  the  belief  that  military  force 
was  an  appropriate  policy  instrument  for  containing  Soviet  expansion.  One  doctrine  that 
was  selected  to  support  these  strategic  objectives  was  the  employment  of  Air  Power. 
However,  the  strategic  environment  has  changed.  William  Cohen  in  his  Testimony 
before  the  Senate  Armed  Services  Committee  on  February  5,  1998  said,  “  The  future 
conflict  environment  will  present  challenges  that  are  dramatically  different  from  those 
confronted  by  our  armed  forces  during  the  Cold  War  and  even  the  Persian  Gulf  War. 
Consequently  our  current  concept  of  tactical  air  forces  will  likely  have  to  change 
dramatically.” 

Presently,  the  strategy  involves  minimizing  serial  production  runs  of  new  aircraft 
under  all  but  two  circumstances:  first,  when  the  new  system  offers  a  major  boost  in 
military  effectiveness  that  solves  a  major  strategic  or  operational  problem,  and  second, 
when  existing  systems  have  reached  the  end  of  their  useful  lives. 
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Doctrine  refers  to  the  operating  principles  of  an  organization  that  link  weapon 
application  to  a  specific  end.  Doctrine  translates  the  generalities  of  strategy  to  operating 
principles.  For  example  the  Air  Force  Modernization  Planning  Process  (AFMPP)  is  part 
of  the  Air  Power  doctrine  and  the  mechanism  for  supporting  the  five  core  competencies- 
air  superiority,  space  superiority,  precision  employment,  global  mobility  and  information 
dominance.  The  AFMPP  integrates  the  elements  that  provide  the  foundation  for  the  five 
competencies  into  a  coherent  modernization  plan  that  reaches  25  years  into  the  future. 
The  foundation  elements  included  in  the  modernization  plan  are: 

•  readiness  and  sustainment; 

•  research,  development,  test  and  evaluation; 

•  logistics; 

•  information  technology; 

•  equipment  and  facilities; 

•  manpower. 

However,  replacement  of  aircraft  has  slowed  significantly  in  recent  years  because 
of  budget  constraints  and  affordability.  The  result  has  been  a  shift  to  increased  upgrading 
and  life  extension  of  many  systems  beyond  what  was  typically  done  in  the  past. 
(Butowski,  p.  3) 

In  an  ideal  world,  strategy  and  doctrine  ought  to  provide  directions  for  persons 
who  are  in  charge  of  design  and  procurement  of  weapon.  But  strategy  and  doctrine  are 
abstractions,  which  do  not  translate  easily  into  the  precise  requirements  for  weapons.  So, 
there  are  other  reasons  that  can  explain  aircraft  modernization  choice. 
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2. 


Readiness  and  Effectiveness 


One  of  the  earliest  paradigms  in  the  literature  of  social  science  is  the  arms  race 
typology.  This  view  to  modernization  was  originally  devised  by  Lewis  Fry  Richardson 
who  developed  models  of  arm  acquisition  with  the  hope  that  understanding  the  process 
might  help  in  preventing  war.  (Sorenson,  p.  39)  The  fundamental  assumption  behind 
arms  race  models  is  that  two  hostile  nations  will  arm  themselves  completely,  with  one 
nation  stimulating  the  arming  of  the  second  in  a  reciprocal  fashion.  Arm  races  are  thus 
“action-reaction  behaviors,”  with  each  nation  trying  either  to  win  the  arms  race  at  best  or 
at  least  is  trying  to  avoid  losing  it.  Nations  as  a  result,  arm  themselves  for  external 
reasons,  reacting  to  a  perceived  threat. 

Strategic  and  structural  categories  of  arms  races  influence  the  technical  level  of 
the  modernization  process,  through  the  demands  of  military  readiness  and  individual  or 
group  effectiveness.  Joint  Pub  1-02  defines  readiness  as  “the  ability  of  forces,  units, 
weapon  systems  or  equipment  to  deliver  the  outputs  for  which  they  were  designed 
(includes  the  ability  to  deploy  and  employ  without  unacceptable  delays).” 

As  technical  element  of  modernization,  readiness  and  effectiveness  have  time, 
range,  loading  and  accuracy  components.  For  example  Russia’s  program  of  tactical 
aviation  development  will  include  the  following  directions:  prolongation  of  the  service 
time,  modernization  and  improvement  of  the  currently  used  fighters  and  missile 
armament  to  broaden  their  ability  to  destroy  both  the  aerial  and  ground  targets,  and 
improving  of  range  and  autonomy.  The  next  direction  is  standardization  of  the  fighter 
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aviation  units  training  within  the  framework  of  Air  Force  and  Air  Defense  formations  to 
provide  both  air  defense  and  battlefield  support.  (Butowski,  p.  7) 

The  essential  aims  of  Russia’s  Air  Force  modernization  are  as  follows: 

•  an  increase  in  combat  effectiveness,  practically  due  to  the  application  of 
new  weapons  previously  prepared  for  the  next  generation  aircraft; 

•  a  multi-role  capability  in  combat  aircraft  obtained  through  the  use  of  new 
fire  control  and  weapon  systems; 

•  an  enhanced  night  and  adverse  weather  mission  capability; 

•  an  expansion  of  the  “information  field”  for  aircrew  through  the  use  of  new 
navigation  systems,  data  links,  etc;  and 

•  a  reduction  in  maintenance  cost  during  the  prolonged  service  life  of 
aircraft  by  replacing  standard  maintenance  schemes  with  maintenance 
according  to  the  actual  technical  conditions  of  aircraft. 

The  results  of  modernization  in  Russia  are  characterized  by  the  growth  of  the 
integral  parameters,  such  as  the  coefficient  of  the  combat  potential  while  attacking  the 
aerial  or  ground  targets.  The  Mikoyan  design  bureau  representatives  presenting  MiG- 
29SMT  program  tell  that  modernized  MiG-29  combat  efficiency  in  an  air-to-air  role  will 
be  2.1  :  1  compared  with  the  current  MiG-29  (  and  even  6.5  :  1  in  long  distance  air 
combat).  The  combat  potential  of  the  aircraft  in  air  to  ground  mission  will  supposedly  be 
increased  by  a  factor  of  3.8  ,  or  even  8  against  point  targets.  (Polushkin,  p.  6) 


3.  Flight  Safety 

Flight  safety  has  a  special  position  among  the  reasons  and  objectives  of 
modernization.  It  is  only  one  indisputable  reason  because  of  the  value  of  the  pilot’s  life. 
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and  threat  of  losing  highly  expensive  weapons  due  to  technical  problems  make  the 
military  and  the  producers  cooperate  to  improve  safety  and  reliability. 

When  flight  safety  design  flaws  are  discovered,  the  consequence  is  usually  the 
suspension  of  operations.  Designers  and  producers  use  all  their  resources  to  eliminate  or 
to  neutralize  possible  threats.  If  flight  restrictions  don’t  correct  a  flight  hazard,  the 
design  bureau  has  to  make  structural  changes  in  the  shortest  possible  time.  This  has  been 
labeled  the  so-called  “urgent  modernization.” 

Another  level  of  modernization  relevant  to  flight  safety  is  pre-planned  or 
preventive  upgrade.  Employment  of  aircraft  reveals  many  weak  or  problematic  areas. 
Sometime  they  are  not  urgent  or  occur  infrequently.  Analyzing  defects  and  malfunctions 
at  the  aircraft  design  bureaus  creates  upgrade  programs  for  regimental  and  depot  levels  of 
maintenance.  The  regimental  specialists  usually  change  specific  details  and  elements, 
while  depot  personnel  traditionally  make  structural  changes.  Producer  specialists  or  even 
design  engineers  can  make  highly  complex  upgrade  and  modifications. 

Soviet  era  statistics  show  that  for  tactical  aircraft,  such  as  the  MiG-25PD,  the 
number  of  elements  to  be  upgraded  would  generally  total  10  to  15  after  each  1500  flight 
hours.  At  the  same  time,  from  1980  to  1988  flights  of  MiG-25’s  were  suspended  six 
times  and  flights  were  restricted  until  structural  changes  were  made  more  than  15  times. 
The  cost  of  a  safety  related  modernization  might  represent  from  8  to  10  %  of  the  lifetime 
aircraft  maintenance  cost. 
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4. 


Obsolescence 


Examples  of  modernization  mainly  caused  by  obsolescence  are  the  Soviet  MiG- 
21  and  the  American  F-5  upgrade  programs.  These  also  are  examples  of  modernization 
that  were  created  to  satisfy  customer  needs.  Countries,  such  as  Romania,  Egypt,  Turkey 
and  Taiwan,  which  possess  these  aircraft  do  not  have  sufficient  funds  to  buy  new  or  even 
used  aircraft  and  have  to  use  available  airframes.  In  both  cases,  design  bureaus  agreed 
that  these  aircraft  are  still  in  good  condition  and  their  useful  lifetime  can  be  prolonged  for 
10  to  15  more  years.  Another  reason  these  countries  can  employ  these  used  aircraft  is 
that  the  air  industry  has  created  a  lot  of  universal  systems  and  elements  that  can  be  easily 
adapted  to  any  airplane.  Examples  are  the  Global  Position  System  (GPS)  navigation, 
radar  and  universal  weapon  delivery  systems.  On  the  other  hand  air  forces  of  many 
countries  have  numerous  special  tasks  (patrolling,  training,  counter  terrorists  operations 
and  so  on)  that  allow  using  aircraft  with  comparably  low  performance.  So,  the  primary 
problem  is  the  obsolescence  of  existing  armament  and  equipment.  From  material 
published  in  aviation  journals,  we  know  that  in  both  cases  (MiG-21  and  F-5)  objects  of 
modernization  are  cockpit  equipment,  navigation,  communication,  radar  and  armament. 

The  advantage  of  such  modernization  is  a  high  level  of  cost  effectiveness  index. 
For  example  the  cost  of  the  MiG-21-98  upgrade  program  is  about  30%  of  an  aircraft’s 
initial  cost  in  constant  rubles,  and  the  estimated  increase  of  effectiveness  is  from  1.7  to 
2.5.  The  problem  is  that  the  technological  and  information  revolution  dramatically 
increases  “the  depreciation  rate”  of  new  equipment  and  weapon  systems  and  as  a  result 
the  comparative  effectiveness  of  old  aircraft  may  be  worse  than  predicted. 


B.  CONSTRAINTS 


1.  Technical  and  Technological 

There  are  two  fundamental  requirements  to  be  fulfilled  before  any  aircraft  can  be 
upgraded  or  new  equipment  can  be  installed.  First  and  foremost,  the  basic  airframe  must 
be  sound  with  an  acceptable  service  life  remaining.  The  airframe  structure  can  deteriorate 
depending  on  the  nature  of  its  use  and  the  level  of  maintenance  it  has  received. 

Second,  if  the  airframe  is  in  good  condition  and  well  maintained,  before 
proceeding  with  the  upgrade,  it  is  important  to  ensure  an  adequate  availability  of  basic 
spare  parts.  If  the  storage  of  spare  parts  is  sparse,  it  is  necessary  to  estimate  the 
possibility  of  ordering  parts  from  the  original  or  side  producer.  Both  options  could  be 
expensive  and  could  significantly  increase  maintenance  cost. 

One  more  important  issue  at  the  present  time  is  the  movement  from  analog  to 
digital  electronic  systems.  For  old  aircraft,  a  partial  upgrade  can  create  two  incompatible 
electronic  systems. 

The  next  problem  is  adapting  new  weapons  to  the  airframe.  Sometimes  the 
weight  of  additional  frame  elements  to  adjust  to  the  new  weapon  to  an  old  aircraft  may  be 
higher  than  the  weight  of  the  additional  weapon  itself.  This  increase  in  weight  might  lead 
to  another  problem:  insufficient  power.  Of  course,  experience  also  suggests  that  many 
systems  could  be  lighter  than  the  ones  they  replace.  However,  aircraft  generally  gain 
weight  during  upgrades,  even  if  this  is  in  the  form  of  additional  weapons-carrying 
capability.  Obviously,  ascertaining  whether  the  engine(s)  will  have  sufficient  power  to 
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deliver  an  equal  or  better  thrust-to-weight  ratio  in  the  upgraded  aircraft  is  important. 
(Gething,  p.  5.) 

There  are  several  arguments  concerning  how  technology  influences 
modernization.  On  the  one  hand  strategy  and  doctrine  may  guide  the  technology  of 
weapon  development,  stimulating  technology.  On  the  other  hand,  it  has  also  been  argued 
that  technology  itself  simulates  and  constrains  the  development  of  strategy,  since 
technological  development  can  make  new  strategies  possible.  Yet  another  argument  is 
that  technology  reduces  uncertainty  and  its  use  may  be  maximized  without  specific 
knowledge  about  the  present  or  future  state  of  adversary  capability.  New  technology  is 
sometimes  developed  independently  from  weapon  projects,  but  then  incorporated  into 
them  because  it  is  available. 

2.  Finance 

In  the  year  2054,  the  entire  defense  budget  will  purchase  just  one  aircraft. 

This  aircraft  will  have  to  be  shared  by  the  Air  Force  and  the  Navy  3.5 

days  each  per  week  except  for  leap  year,  when  it  will  be  made  available  to 

the  Marines  for  the  extra  day. 

Norman  R.  Augustine 

As  economic  difficulties  impact  a  world  already  wrestling  with  reduced  defense 
budgets,  the  hard  fact  is  that  fewer  new  weapons  platforms  are  being  developed.  Yet  it  is 
not  only  the  budgetary  impact  that  is  decreasing  this  development.  The  state’s  military 
budget  expands  all  security  problems  including  strategy  and  force  structure. 

Balancing  fiscal  constraints  against  mission  requirements  is  a  relatively  new 
problem  for  military  planners  in  the  former  Soviet  republics.  New  democratic  processes 
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dictate  that  the  militaiy  compete  for  funding  along  with  the  other  needs  of  the  nation.  In 
the  1980s  the  Soviet  Union  spent  about  20%  of  the  GNP  on  the  military,  while  the 
Ukraine  presently  can  spend  only  2%  of  the  GNP.  Money  available  for  military 
programs  is  significantly  different  from  that  ones  available  ten  years  ago,  and  so  are 
possibilities  of  military  industry.  We  can  expect  this  state  of  affairs  to  continue  in  the 
future. 

Even  so,  after  the  Gulf  War  and  the  Serbia  conflict  the  world  was  convinced  of 
the  importance  of  air  power  in  the  modem  conflicts;  however,  any  nation  can  spend  only 
a  reasonable  amount  of  money  for  defense.  The  age-old  question  of  “How  much  is 
enough?”  will  surely  continue. 

As  we  look  at  the  cost  of  modernization,  high  technology  has  an  attractive  lure. 
Technology  offers  high  military  efficiency,  stability  and  economy.  But  at  some  point  in 
the  development  of  a  weapon  system,  the  additional  cost  of  increased  capability 
outweighs  the  gains. 

3.  Scientific  and  Industrial  Base 

When  analyzing  particular  aircraft  modernization  programs,  it  is  easy  to  say  that 
the  main  role  in  definition  of  such  a  program  belongs  to  the  original  design  bureau  and 
manufacturer.  Two  scenarios  are  possible.  First  is  the  modernization  of  domestic  aircraft, 
when  all  stages  of  an  aircraft  lifecycle  belong  to  country-owner.  This  is  the  simplest  case 
and  this  modernization  is  usually  constrained  only  by  a  lack  of  money  or  technology. 
Second  is  when  the  owner  bought  or  inherited  aircraft  without  previously  agreed 
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modernization.  In  this  situation,  modernization  is  also  strongly  constrained  by  the 
relations  with  the  designer  and  manufacturer. 

Even  if  the  country  has  a  modem  scientific  and  industrial  base,  it  can’t  modernize 
significantly  without  cooperating  with  the  design  bureau.  The  design  bureau  owns  the 
technical  information  concerning  the  particular  aircraft,  and  foreign  customers  might 
select  only  those  programs  proposed  by  a  designer,  or  they  might  order  a  new  project. 
Such  an  upgrade  program  may  require  much  more  money  and  time. 

The  design  bureau  is  best  qualified  to  estimate  the  lifetime  for  the  frame  and 
elements.  It  studies  possibilities  to  change  systems  and  armament  and  finally  produces 
an  upgrade  program.  Examples  of  the  MiG-21  and  MiG-29  modernization  show  that  the 
initial  intentions  of  some  nations,  even  those  with  modem  air  industry,  to  undertake  their 
own  modernization  finally  led  to  a  close  cooperation  with  the  original  designer.  The 
main  reason  for  the  design  bureau’s  leadership  is  that  it  possesses  full  technological  and 
construction  documentation.  Any  changes  made  by  another  firm  may  weaken  the 
design  or  introduce  incompatible  elements  in  operation.  If  uncoordinated  changes  are 
made,  the  design  bureau  can  deny  any  responsibility  for  future  flight  safety. 

Another  constraining  factor  for  countries  that  cannot  produce  their  own  aircraft  is 
an  insufficient  industrial  base  for  providing  an  upgrade.  For  example,  all  Warsaw  pact 
countries  inherited  air  depots  able  to  provide  high  quality  maintenance  and  repairs. 
Depending  upon  the  level  of  design  changes,  modernization  can  be  made  by  a  depot  (as 
in  the  case  of  the  MAPO-  D  ASSO  program  for  MiG-29)  or  by  a  main  plant  (as  in  the 
case  of  the  MiG-29SMT  modernization).  Clearly,  modernization  conducted  by  a 
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domestic  depot  is  cheaper  and  gives  additional  advantages,  such  as  professional  repair 
personnel  and  the  relative  independence  from  manufacturer  in  future  aircraft 
maintenance. 

C.  DECISION  MAKING  METHODOLOGIES 

What  makes  decisions  hard?  Certainly  different  programs  may  involve  unique 
difficulties.  For  example,  any  modernization  decision  requires  us  to  think  about  the 
interests  of  various  groups,  as  well  as  to  consider  the  limitations  of  information  on  the 
inputs  and  outputs.  Although  every  decision  may  have  its  own  special  problems,  there 
are  four  basic  sources  of  difficulty.  (Clemen,  p.  2)  First,  a  decision  can  be  hard  simply 
because  of  its  complexity.  In  case  of  modernization,  an  Air  Forces  must  consider  many 
individual  issues:  prices,  effectiveness,  lifetime,  industry  base,  etc.  Second,  a  decision 
can  be  difficult  because  of  the  inherent  uncertainty  in  the  situation.  Third,  a  decision¬ 
maker  may  be  interested  in  working  toward  multiple  objectives,  even  though  progress  in 
one  direction  may  impede  progress  in  others.  Finally,  a  problem  may  be  difficult  if 
different  perspectives  lead  to  different  conclusions.  Or,  even  from  a  single  prospective,  a 
slight  change  in  certain  inputs  may  lead  to  different  choices.  Different  individuals  may 
look  at  the  problem  from  different  prospective,  or  they  may  disagree  on  the  uncertainty  or 
value  of  the  different  outcomes. 

So,  which  methodologies  exist  to  help  us  make  a  decision?  They  are  legion. 
Different  schools,  approaches  and  theories  give  us  a  large  selection  of  scientific  methods 
for  decision  making  and  support.  The  military  planners  use  four  main  groups  of 
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methods:  benefit-cost,  cost-effectiveness,  cost-utility  analysis  and  dynamic  modeling. 
Let’s  examine  the  main  features  of  these  four  groups. 

1.  Benefit-Cost  Analysis 

Benefit-cost  analysis  is  recommended  by  the  US  Office  of  Management  and 
Budget  (OMB)  as  the  technique  to  use  in  a  formal  economic  analysis  of  Government 
programs  or  projects.  According  to  the  OMB  Circular  A-94  “Guidelines  and  Discount 
rates  for  Benefit-Cost  Analysis  of  Federal  Programs,”  benefit-cost  analysis  (BCA)  is  used 
“to  support  governmental  decision  to  initiate,  renew,  or  expand  programs  or  projects 
which  would  result  in  a  series  of  measurable  benefits  or  cost  extending  for  three  or  more 
years  into  the  fixture.” 

The  standard  criterion  used  in  BCA  is  the  net  present  value  (NPV)  the  discounted 
monetized  value  of  expected  net  benefit.  The  NPV  is  computed  by  assigning  monetary 
values  to  benefits  and  costs,  discounting  future  benefits  and  costs  using  an  appropriate 
discount  rate,  and  subtracting  the  sum  total  of  discounted  cost  from  the  sum  total  of  the 
discounted  benefit.  Discounting  benefits  and  costs  transforms  gains  and  loses  occurring 
at  different  times  to  a  common  unit  of  measurement.  Circular  A-94  states  that  programs 
with  a  positive  NPV  increase  social  resources  and  are  generally  preferred.  Programs  with 
a  negative  NPV  should  generally  be  avoided. 

The  main  problem  of  this  method  is  that  the  NPV  is  not  always  computable 
because  monetary  values  of  some  benefits  and  costs  cannot  be  determined.  To  overcome 
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uncertainties  in  the  BCA  Circular  94A  proposes  to  use  comprehensive  enumeration  and 
quantifying  of  benefits  and  costs. 

The  main  elements  of  the  BCA  are: 

•  Policy  rationale.  A  rationale  should  be  clearly  stated  in  the  analysis; 

•  Explicit  assumptions.  The  analysis  should  include  a  statement  of  the 
assumptions,  the  rationale  behind  them,  and  the  review  of  their  strengths 
and  weaknesses; 

•  Evaluation  of  alternatives  by  means,  by  different  program  scales,  by 
methods  of  provision  and  by  different  degrees  of  government 
involvement; 

•  Verification  of  determination  whether  all  anticipated  benefits  and  costs 
have  been  realized. 

Those  working  with  the  BCA  should  consider  such  variables  as  incremental 
benefits  and  costs,  interactive  and  international  effects,  inffamarginal  and  indirect 
benefits  and  costs.  After  that,  analysts  should  include  calculations  for  inflation  and  risks. 

The  results  from  the  BCA  are  explicit  and  obvious.  However,  difficulties  with  the 
definition  of  benefits  and  costs  of  military  indexes  relevant  to  combat  effectiveness  make 
the  BCA  rarely  used  in  military  planning.  Even  though  the  BCA  has  special  tools  to  deal 
with  uncertainties,  the  huge  number  of  those  uncertainties  makes  it  extremely  difficult  to 
use  in  real  tasks. 

Three  limitations  of  the  BCA  are  common.  First,  analysts  may  be  unwilling  or 
unable  to  monetize  the  most  important  policy  impacts.  Second  is  that  any  particular 
effectiveness  measure  does  not  capture  all  of  the  social  benefits  of  each  alternative. 
Finally,  analysts  frequently  deal  with  intermediate  goods  which  linkage  to  preferences  is 
not  clear. 
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In  such  cases,  commonly-used  alternatives  to  the  BCA  are  cost  effectiveness  and 
cost-utility  analysis. 

2.  Cost-Effectiveness  Analysis 

If  all  alternatives  are  mutually  exclusive,  and  the  status  quo  is  among  the 

alternatives,  sharing  similar  scales  and  patterns  of  cost  and  benefit,  then 

cost  effectiveness  analysis  does  select  the  most  effective  policy. 

(Boardman,  p.  396) 

OMB  opinion  holds  that  cost-effectiveness  analysis  (CEA)  is  less  than  a 
“comprehensive  technique,  but  it  can  be  appropriate  when  the  benefits  from  competing 
alternatives  are  the  same  or  where  a  policy  decision  has  been  made  that  the  benefits  must 
be  provided.” 

According  to  the  OMB  Circular  NO  A-94  a  program  is  cost  effective,  if  on  the 
basis  of  life  cycle  cost  analysis  of  competing  alternatives,  the  program  is  determined  to 
have  the  lowest  cost  expressed  in  present  value  terms  for  a  given  amount  of  benefits. 
Cost  effectiveness  analysis  is  appropriate  whenever  it  is  unnecessary  or  impractical  to 
consider  the  dollar  value  of  the  benefits  provided  by  the  alternatives  under  consideration. 
This  is  the  case  whenever  (1)  each  alternative  has  the  same  annual  benefits  expressed  in 
monetary  terms;  or  (2)  each  alternative  has  the  same  annual  effects,  but  a  dollar  value 
cannot  be  assigned  to  their  benefits.  An  analysis  of  alternative  defense  systems  often 
falls  in  this  category. 

In  comparison  with  the  BCA,  cost-effectiveness  analysis  produces  a  ranking  but 
does  not  provide  explicit  information  about  whether  there  would  be  positive  net  social 


20 


benefits  associated  with  any  of  the  alternatives  being  considered.  Since  CEA  does  not 
monetize  benefits,  it  inevitably  involves  two  different  metrics:  monetary  cost  and  an 
effectiveness  measure.  Because  of  the  nature  of  these  metrics  and  scale  differences  one 
cannot  add  or  subtract  one  from  another.  Thus  it  is  a  measure  of  economic  efficiency. 

In  practice,  CEA  almost  inevitably  reduces  negative  impacts  that  would  be 
included  in  BCA.  Indeed,  CEA  typically  considers  only  the  measures  of  effectiveness. 
Military  projects  usually  have  multiple  objectives  and  benefits.  Moreover,  the  necessity 
of  obtaining  an  arbitrary  ranking  increases,  as  alternatives  become  less  similar  in  terms  of 
the  inputs  and  outputs.  From  this  point  we  enter  the  area  where  cost-utility  and  multi¬ 
attribute  utility  analysis  give  the  best  result. 

3.  Cost-Utility  Analysis 

Cost-utility  analysis  (CUA)  relates  budgetary  cost  to  a  single  benefit  measure,  but 
its  benefit  measure  is  a  construct  composed  of  several  benefit  categories.  Boardman 
believes  that  “CUA  can  be  thought  of  as  a  form  of  CEA  employing  a  more  complex 
effectiveness  measure.”  (p.  403)  As  CUA  involves  two  or  more  distinct  variables  the 
analyst  must  designate  how  these  variables  are  to  be  defined  and  combined.  This  is  a 
problem  in  multi-attribute  decision  making. 

Utility  analyses  are  useful  in  two  important  areas:  (1)  to  quantify  attributes  that 
don’t  have  an  obvious  unit  of  measurement,  and  (2)  to  quantify  and  model  a  decision¬ 
maker’s  propensity  to  accept  or  to  avoid  risk.  (Marshal,  p.  249)  The  most  difficult  part 
of  such  an  analysis  is  to  obtain  a  utility  function.  Sometimes  it  is  unrealistic,  unclear  or 
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too  risky.  But  this  does  not  negate  the  usefulness  of  this  procedure  in  decision  modeling. 
Indeed,  the  process  of  utility  function  construction  often  gives  insight  and  clarity  to  the 
problem. 

There  are  two  ways  to  determine  a  utility  function  when  decisions  are  made  under 
conditions  of  uncertainty.  One  is  when  an  indifference  probability  is  found  for  a  fixed 
risky  venture.  Another  way  to  determine  a  utility  function  is  when  a  riskless  alternative 
payoff  is  found  that  is  equivalent  to  the  expected  payoff  of  a  given  risky  venture  with 
even  odds  of  success. 

At  a  strategic  level  of  military  planning,  the  effectiveness  measure  for  a  decision 

problem  may  be  captured  by  criteria  such  as  readiness,  sustainability  or  force  projection. 

Some  attempts  to  model  these  problems  have  an  inherent  but  usually  unstated, 

assumption  that  because  no  measurement  units  exist,  the  attributes  can  be  represented  by 

dimensionless  quantities.  To  prevent  inconsistencies  in  model  output,  Kneale  Marshal 

states  and  uses  the  following  principle  as  a  fundamental  guide  in  model  building: 

For  a  multi-attribute  decision  model  to  be  consistent  it  should  apply  the 
same  rules  for  combining  attributes  that  cannot  be  measured  directly  as  it 
does  for  those  that  can.  If  the  problem  under  consideration  has 
performance  attributes  for  which  there  are  no  obvious  measurement  units, 
one  should  not  assume  that  the  weights  assigned  to  these  attributes  are 
dimensionless  and  hence  can  be  normalized  in  an  arbitrary  manner. 


4.  Dynamic  Modeling 

Computers  allow  analyst  to  make  huge  calculations  rapidly.  Computers  also  allow 
them  to  use  more  and  more  complicated  models  in  simulation  of  real  processes.  There 
are  many  different  models  for  simulation  of  military  actions  and  even  regional  wars. 
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Many  military  planners  and  scientists  have  found  these  models  useful  for  resource 
allocation  planning.  Multi-level  dynamic  models  can  simulate  reality,  changing  inputs  in 
accordance  with  a  given  scenario  and  state  of  outputs.  Modem  models  of  global  and 
local  military  conflicts  are  based  on  complicated  probabilistic  scenarios  and  complex 
mathematical  theories.  These  models  let  researchers  evaluate  the  influence  of  entire 
weapon  systems,  or  just  their  parameters,  on  the  result  of  simulated  military  operations. 
Clearly  the  quality  of  the  results  in  dynamic  modeling  depends  strongly  on  the  details  of 
the  model,  the  quality  of  assumptions  and  simplification.  The  results  received  in 
different  stages  of  simulation  may  then  be  used  to  construct  specific  cost-effectiveness  or 
cost-utility  indexes  for  decision  making. 

For  example  the  DynaRank  methodology  (Hellestad,  p.  7)  can  work  with  multiple 
objectives  and  rank  policy  options  by  cost-effectiveness.  Each  ranking  is  a  judgement 
about  the  relative  importance  of  higher  level  objectives  and  a  variety  of  success  criteria. 
It  can  also  be  used  to  integrate  a  detailed  analysis  with  emphasis  on  components  of 
defense  strategy. 

On  the  other  hand,  complicated  models  have  to  work  with  stochastic  elements 
involved  in  the  simulations.  These  means  that  every  new  run  can  give  different  results, 
and  repeated  simulation  runs  may  result  in  different  sets  of  summary  performance 
measures. 
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B.  ANALYSIS  OF  ALTERNATIVE  MEASUREMENT  SCALES 

Decision-makers  deal  with  four  different  levels  of  measurement:  nominal,  ordinal, 
interval  and  ratio.  Money,  often  the  main  index  in  analysis,  has  ratio  measurement. 
Analysts  always  try  to  work  with  compatible  indexes  when  assessing  different  policy 
alternatives.  In  this  case  it  would  be  easy  to  say  that  doubling  commodities  doubles 
money  cost.  Unfortunately,  when  we  use  effectiveness  or  utility  as  indexes  in  analysis,  it 
is  sometimes  very  difficult  to  make  the  right  decision  due  to  the  scale  differences. 

In  simple  cases,  when  all  of  the  policy  alternatives  have  the  same  cost,  scale 
differences  do  not  cause  a  problem.  If,  in  addition,  the  cost-effectiveness  analysis  is 
inclusive  of  all  social  costs  and  benefits,  then  CEA  ranks  alternatives  in  terms  of 
allocative  efficiency.  Similarly,  scale  is  not  a  problem  if  the  level  of  effectiveness  is 
constant  across  all  alternatives.  In  case  of  fixed  effectiveness,  CEA  corresponds  to  a 
simple  cost  minimization  problem,  while  in  the  fixed  budget  case  CEA  corresponds  to  a 
simple  effectiveness  optimization  problem. 

In  real  problems  we  can  find  that  large  scale  differences  among  alternatives  can 
distort  choice.  For  example,  we  have  two  exclusive  alternatives.  In  the  first  case  we  can 
invest  $10  million  in  flight  safety  and  save  ten  lives  a  year.  The  second  alternative  will 
cost  $200  million  and  would  be  able  to  prevent  100  military  casualties  in  a  conflict.  Due 
to  the  simple  cost  effectiveness  ratio,  the  first  alternative  is  twice  as  effective  as  the 
second.  But  the  second  alternative  can  save  ten  times  more  lives  in  certain  conditions. 

A  different  problem  arises  if  the  decision-maker  must  select  between  indexes  that 
are  in  different  measurement  scales.  An  example  of  such  a  problem  is  how  to  select 
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between  the  ordinal  ranking  of  some  radar  qualities  and  the  ratio  measures  of  additional 
weight  relevant  to  this  change.  Different  methodologies  propose  different  approaches  to 
convert  scale  and  measurement  differences  to  compatible  results. 

The  first  method  is  normalization.  TASCFORM  methodology,  for  example,  uses 
the  F-4B  as  a  basis  for  comparing  different  aircraft.  Then  it  computes  the  ratios  of  all 
performances  of  evaluated  aircraft  to  F-4B’s  performances.  The  methodology  effectively 
constructs  a  ratio  scale  in  terms  of  F-4B  equivalents,  but  the  constructing  scoring  system 
doesn’t  determine  the  utility  of  F-4B  equivalents. 

Utility  theory  proposes  another  way  to  solve  a  measurement  problem.  Single 
utility  functions  (SUF)  let  us  transform  differently  measured  attributes  to  a  common  unit 
called  “utility.”  Actually  SUFs  play  the  role  of  normalization  of  different  measures  to 
common  measures  with  interval  values.  Individual  trade-off  between  attributes,  in 
different  methodologies  that  use  utility  theory,  are  usually  based  on  personal  judgement. 
In  this  case  we  receive  an  interval  ranking  of  alternatives. 

Probabilities  and  uncertainties  when  added  to  analysis  may  change  scales  of 
results.  As  a  result,  when  the  lottery  option  discussed  above  is  used,  one  can  obtain  a 
scale  that  has  interval  properties  in  the  sense  that  linear  transformations  of  the  utility 
function  can  be  taken  without  changing  the  decision.  However,  one  really  obtains  an 
ordinal  scale  for  utility  when  the  developed  utility  function  is  used  to  assess  alternative 
combination  of  utility  and  some  other  variable,  such  as  cost,  is  not  included  in  the  utility 
function. 
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III.  CASE.  MIG-29  MODERNIZATION  DECISION 


The  ultimate  goal  of  military  economics  is  to  discover  the  truth  about  relations 
between  military  goals  and  the  economy  in  the  real  world.  There  are  many  ways  to  seek 
the  truth.  Some  scholars  believe  in  empirical  verification  as  evidence  supported  by 
theory;  others  assert  that  truth  can  be  discovered  only  by  logical  deduction.  The  methods 
may  vary,  but  the  common  element  is  the  reliance  on  models.  A  situational  model  for 
methodology  evaluation  is  an  independent  element  of  analysis.  It  can  be  used  for  all 
evaluated  methodologies  without  sufficient  changes.  Moreover  such  a  model  should  be 
“method  neutral,”  so  it  shouldn’t  influence  the  final  results  by  giving  advantages  to  any 
methodology.  The  best  index  of  effectiveness  for  a  modernization  decision  is  Force 
Potential.  This  means  that  a  model  should  have  Force  Potential  as  an  output. 

A  common  organizational  model  that  satisfies  most  present  demands  is  the 
McCaskey  model.  (Figure  1).  First,  this  model  includes  all  the  factors  that  can  influence  a 
modernization  decision.  Second,  this  model  describes  the  interconnections  of  the 
elements.  Finally,  this  model  allows  us  to  use  integrated  indexes  such  as  force  potential 
for  analytical  purposes.  But,  this  model  is  too  broad  for  numerical  analysis.  Practical 
models  are  usually  derived  from  methodology  and  analysis  demands.  So,  what  are  the 
main  features  of  the  Ukrainian  Air  Force  that  should  be  under  considered  for 
modernization  modeling? 
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Figure  1.  Air  Force  Organizational  Model 

A.  ENVIRONMENT  AND  DIRECTIONS 

The  main  elements  of  the  external  environment  of  the  Ukrainian  Air  Force  are  the 
State,  its  security  formations,  politics  and  the  economy.  Strategy  and  doctrine  set  the 
main  directions. 

Ukraine  is  a  non-nuclear  state  and  is  defensive  in  character.  At  a  doctrinal  level, 
the  Ukraine  does  not  recognize  territorial  claims  upon  itself  or  hold  territorial  claims  on 
other  states.  The  technical  aspect  of  military  doctrine  emphasizes  defense  sufficiency. 

Guided  by  the  principles  of  non-participation  in  military  blocs  and  alliances, 
which  the  country  remains  faithful  to,  the  Ukrainian  army  is  now  prepared  to  fight  only 
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on  its  own  territory.  This  really  means  that  the  Ukraine,  unlike  many  European  NATO 
countries,  cannot  afford  a  small  or  a  weak  army. 

The  Ukrainian  military  for  the  last  five  years  have  implemented  territorial 
principle  of  defense  strategy  and  new  military  administrative  organization.  The  next  step 
of  the  military  reform  will  be  reorganizing  divisional  structure  to  a  brigade-battalion 
system.  The  Air  Force  will  shrink  to  10-12  brigades.  In  addition  600  aircraft  and  400 
helicopters  will  be  transferred  from  combat  units  to  reserve  or  will  be  disassembled. 

The  state  intends  to  support  a  technical  level  of  modernization  with  thorough 
modernization  of  its  existing  weapons.  The  state  weapon  program  announced  that  the 
lifetime  of  the  main  armament  will  be  increased  to  10-15  years.  Also  the  procurement  of 
new  weapons  will  be  minimized. 

Financial  support  of  the  army  is  insufficient.  Ukraine’s  military  budget  for  the 
year  of  2000  is  Hr2.4  billion  ($480  mil.  One  dollar  approximately  equals  five  hrivnas.) 
Objective  data  shows  that  for  the  last  several  years  the  army  has  been  underfinanced. 
Hr  2.4  billion  military  budget  in  existing  conditions  permits  weapon  R&D  plus 
procurement  in  the  level  of  approximately  Hr300  millions.  ($60  min.)  So,  one  can  say 
that  the  Ukrainian  military  has  already  lost  one  important  campaign  -  the  budget  war. 
For  the  year  2000,  they  received  less  than  half  of  what  MOD  analysts  have  estimated  to 
be  their  minimum  needs. 

The  present  external  environment  for  Ukraine  can  be  characterized  as  friendly  or 
neutral.  On  the  other  hand,  all  the  main  neighboring  countries  have  different  historical 
claims  for  different  parts  of  Ukrainian  territory.  The  Ukraine  has  cultivated  a  good 
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relationship  with  NATO.  It  was  the  first  CIS  State  to  join  the  Partnership  for  Peace  (PfP). 
There  are  a  number  of  special  agreements  between  Ukraine,  the  United  States,  Russia  and 
NATO.  These  agreements  were  created  to  increase  state  security  after  Ukraine’s  nuclear 


disarmament. 

Ukraine’s  economic  potential  is  considerable;  it  has  strong  agricultural  traditions, 
a  large  domestic  market  and  trained  labor,  a  diverse  industrial  base  and  substantial  natural 
resources.  However,  the  economic  development  has  been  slow  and  reforms 
inconsistently  applied  and  frequently  subject  to  bureaucratic  interference  at  both  national 
and  local  levels.  The  Ukraine  depends  on  imported  energy  from  Russia  and 
Turkmenistan.  The  foreign  debt  equals  $13.5  billion. 

Owing  to  a  new  state  program,  by  December  31,  1999,  the  Ukrainian  army  will 
have  310,000  serviceman  and  90,000  civilian  employees.  The  state  armament  program 
will  include  R&D  in  the  areas  of  a  new  missile  complex  for  ground  forces,  a  new  anti¬ 
aircraft  complex  for  the  air  defense  and  an  aircraft  modernization  program.  The  division 
of  personnel  and  the  composition  of  Ukrainian  armament  are  given  in  Figures  (2)  and  (3). 
The  Air  Force  structure  is  presented  in  Figure  (4). 
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Figure  2.  Division  of  Personnel  in  Ukrainian  Armed  Forces 
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Figure  3.  Armament  of  Ukrainian  Armed  Forces 


## 

TYPE 

ROLE 

QUANTITY 

IN  SERVICE 

1 

MiG-23  ML,MLD/UB 

Interceptor/Trainer 

100/27 

100/27 

2 

MiG-29-16 

Interceptor 

161 

161 

3 

SU-17 

Ground  Attack 

39 

39 

El 

SU-24MP/MR/U 

Bomber 

160/43/48 

160/43/48 

5 

SU-25 

Close  Ar  Support 

34 

34 

SU-27 

Interceptor 

70 

70 

7 

TU-22/m3 

Bomber 

33/29 

33/29 

8 

An- 12/24/26/  IL-76 

Transport 

40/30/30/188 

40/30/30/188 

i 

Figure  4.  Main  Inventory  of  Ukrainian  Ar  Force 


B.  STRUCTURE  AND  TECHNOLOGY 

Ukraine’s  current  adherence  to  the  CFE  treaty  limits  it  to  1,090  combat  aircraft, 
330  armed  helicopters  and  100  naval  aircraft.  Aircraft  assets  is  divided  75:25  between 
the  Ar  Force  and  the  Ar  Defense. 

According  to  the  new  concept,  the  basic  structure  of  the  Ar  Force  will  be  a 
brigade  consisting  of  4  to  5  squadrons,  and  12  to  14  aircraft  each.  The  brigade  will  have 


31 


one  basic  airfield  where  all  structures  are  based  during  peacetime  and  may  have  up  to 
four  reserve  airfields  for  maneuvers.  The  brigade  might  have  one  or  two  types 
(modifications)  of  the  aircraft  of  the  same  class.  The  normative  ratio  of  pilots  to  aircraft 
is  1.5:1.  Air  Force  norms  say  that  fighter  pilots  must  have  90  tol  10  flight  hours  a  year  to 
maintain  or  increase  their  proficiency.  Presently  the  real  flight  rate  of  Ukrainian  pilots  is 
about  40-50  hours  a  year.  This  is  really  a  small  number  in  comparison  with  NATO’s 
standard  of  150  to  220  flight  hours. 

Each  air  squadron  has  its  own  maintenance  command,  which  is  able  to  work 
autonomously  from  the  base.  Squadron  engineers  and  mechanics  are  able  to  localize  any 
defect  and  to  perform  simple  repairs  and  regulation  tasks.  The  brigade  technical  unit 
provides  periodical  maintenance  (each  100,  200  or  500  of  flight  hours).  The  brigade  is 
also  the  main  unit  for  combat-damage  repair.  Periodical  repair  (1500  to  3000  flight 
hours),  upgrade  and  modernization  are  usually  handled  by  air  depots  -  separate  military 
units  connected  to  center  or  territorial  command.  Every  depot  has  a  narrow  specialization 
according  to  the  types  or  models  of  aircraft.  The  air  industry  of  Ukraine  is  able  to  design 
and  produce  modem  transport  and  civilian  aircraft.  There  are  two  major  air  assembly 
plants,  one  engine  plant  and  several  aggregate,  avionics  and  radio  equipment  plants  in  the 
industry. 

C.  ALTERNATIVES 

The  Ukraine  Air  Force  has  161  MiG-29  fighters.  90  of  them  (two  brigades)  are 
aircraft  model  MiG-29S  (design  bureau  index  9-13)  which  fit  the  MiG-29SMT 
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modernization  program.  All  of  them  have  sufficient  lifetime  and  technical  conditions  to 
be  modernized. 

Let  us  assume  that  the  Ukrainian  MOD  is  in  the  process  of  the  MiG-29 
modernization  program  selection.  The  money  available  for  this  program  for  five  years  is 
$100  millions.  Forty  million  is  earmarked  for  the  first  year  and  $30  million  is 
appropriated  for  the  second  and  third  years  respectively. 

The  Air  Force  command  (1),  The  Air  Force  University  (2)  and  the  General  Staff 
Aviation  Planning  Group  (3)  have  presented  three  different  alternatives: 

•  MIG  MAPO  MiG-29SMT  program  (Air  Force  command).  (MAPO  is 
Russian  abbreviation  for  Moscowskoe  Aviacionnoe  Proizvodstvennoe 
Ob’edinenie); 

•  MAPO-DASSO  program  (Air  Force  University); 

•  Intensifying  training  program  without  major  modernization  (General 
Staff).  ' 

After  the  initial  discussions  are  completed,  the  MOD  military  board  ordered  the 
Armament  Department  to  conduct  a  military-economic  analysis  of  the  alternatives.  As 
the  index  of  effectiveness,  the  experts  needed  to  use  the  integrated  combat  potential  of 
the  Air  Force  brigade  for  five  years.  Analysis  experts  agreed  to  use  two  different  tools: 
(1)  TASCFORM  methodology  (Timperlake,  1-2)  and,  (2)  Logical  Decision  for  Windows 
(LDW).  The  Data  available  for  alternative’s  assessment  are  shown  in  Figure  (5).  The 
qualitative  description  of  proposed  alternatives  is  given  below. 
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N 

PARAMETER 

ALTERNATIVES 

Mapo 

SMT 

Service  life 

Hours 

4000 

5000 

6000 

Max  weapon  load 

kg 

4000 

4500 

Operating  weight  (empty) 

kg 

10900 

11000 

12500 

Max  fuel  load 

kg 

4640 

5600 

Normal  T-0  weight 

kg 

15400 

16000 

Range 

km 

1565 

1565 

2100  ,  i 

Missile  range 

50 

110 

Useful  air  speed 

2440 

Trust 

kn. 

16600 

16600 

Maximum  take  off  weight 

kg 

18500 

19700 

21300 

■ 

Speed 

Max 

2445 

2445 

2445 

1 

S/L 

1500 

1500 

1600 

1 

T/O 

260 

Max  rate  of  cl 

imb  at  S/L 

m/min 

19800 

19800 

21000 

Service  ceiling 

m 

18000 

18000 

18500 

Radius  of 

800  km/h 

350 

Turn  (3.8g) 

408km/h 

225 

■ 

Range 

With  max  internal  fuel 

km 

■ 

1430 

2100 

With  underbelly  tank 

wjf 

2800 

■ 

With  three  fuel  tanks 

3500 

G  limits 

Above  M0.85 

7.5 

7.5 

7.5 

Bellow  M0.85 

9 

9 

9 

Radar 

Detection  range  (fighter) 

100 

130 

Tracked  targets 

10 

10 

Engage 

2 

2 

4 

Air-to-air 

Long  range 

(2)  1 

Mid  range 

2 

2 

4 

Short  range 

4 

4 

4 

■ 

Air-to- 

Unguided  bombs 

+ 

+ 

+ 

■ 

ground 

Guided  bombs 

+ 

■ 

Missiles 

+ 

■ 

Power  plant 

Type  and  number 

2xRD33 

2xRD33 

2  xRD43 

Thrust 

KN 

49.4 

49.4 

98.1 

Thrust  +Afterbum. 

KN 

54.9 

54.9 

Lifetime 

Hours 

900 

900 

1500 

Base  of  avionics 

Specialized 

MIL-STD- 

MIL-STD- 

computer 

1553B 
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Figure  5.  Initial  Data  for  Modernization  Decision 
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1. 


The  MiG-29SMT  Program 


I  assume  that  the  fixed  cost  for  both  the  SMT  and  the  MAPO-DASSO  program  is 
$10  million.  This  includes  the  cost  of  the  technological  documentation,  any  additional 
equipment  and  personnel  training.  The  variable  cost  of  SMT  program  is  $3  million  for 
each  aircraft,  including  the  cost  of  new  aircraft  equipment,  labor  and  overhead.  Lviv’s 
air  depot  estimates  that  it  would  be  able  to  modernize  10  aircraft  a  year  to  the  SMT  level 
after  a  six-month  preparation  period. 

The  main  features  of  the  program  are: 

•  lifetime  prolongation  complex; 

•  increasing  fuel  capacity; 

•  upgraded  radar; 

•  a  new  air-to-air  and  air-to-ground  weapon; 

•  new  countermeasure  equipment; 

•  upgraded  avionics  and  cockpit; 

•  radar  dissipating  coating.  (Polushkin,  p.  6) 

2.  The  MiG-29  MAPO-DASSO  Program 

The  variable  cost  for  the  MAPO-DASSO  modernization  is  $  1  million  for  each 
aircraft.  One  air  depot  after  a  three-month  preparation  period  can  provide  modernization 
of  30  aircraft  a  year.  The  main  features  of  this  program  are: 

•  IFF  transponder  with  all  current  modes; 
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•  A  free  selectable  VHF/UHF  communication  set,  emergency 
communication  and  collision  warning  lights; 

•  Lifetime  enlargement  program  (1 , 100  flight  hours); 

•  GPS  system  with  a  1553B  databus  link  to  an  aircraft  navigation  system, 
plus  an  enhanced  1553B  databus  compatible  system  with  an  integrated 
laser-internal  navigation  system.  (Mader,  p.  3) 

3.  Training  Program 

Contrary  to  groups  presenting  particular  modernization  programs,  the  operational 
planners  propose  to  spend  any  available  money  to  intensify  the  Air  Force  training 
program.  Their  argument  is  based  on  the  fact  that  pilots  don  t  receive  sufficient  training 
and  any  investment  in  aircraft  will  produce  minimum  benefits.  Combat  effectiveness 
depends  on  both  aircraft  performance  and  crew  proficiency,  so  modernization  is  cost 
effective  only  with  a  certain  level  of  pilots  training.  Presently  the  average  flight  time  for 
pilot  is  40  to  50  hours  a  year.  The  data  and  calculations  presented  by  the  General  Staff 
group  show  that  the  best  results  can  be  reached  by  spending  the  available  money  in  the 
following  ways: 

•  aircraft’s  lifetime  prolongation  program  for  five  years  -  $20  million; 

•  spare  parts  -  $15  million; 

•  modem  training  equipment  and  simulators  -  $15  million; 

•  fuel  and  materials  ( $10  mln./year)  x  (5  years)  =  $50  million. 

The  cost  of  fuel  appears  to  be  the  only  variable  cost  for  the  flight-intensifying 
program.  The  Air  Force  should  at  any  rate  spend  money  for  personnel  and  base 
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maintenance  with  or  without  flights.  Costs  of  aircraft  and  equipment  are  sunk  in  this  case 
and  costs  of  other  materials  are  significantly  smaller  than  the  cost  of  fuel. 

Aircraft  fuel  in  the  Ukraine  is  about  $400  1,  000  kilogram.  An  additional  $10 
million  a  year  remains  to  buy  2  5,  000  tons  of  fuel.  MiG-29  consumes  approximately  3 
tons  of  fuel  in  a  one-hour  flight.  This  gives  the  Air  Force  an  additional  8,300  flight 
hours.  For  each  of  130  pilots  of  two  MiG-29  brigades  this  cost  roughly  provides  65 
additional  flight  hours  per  year  for  five  years.  Together  with  the  available  flight  time  this 
number  will  reach  about  100  hours  a  year  per  pilot. 


Pilot  Proficiency  as  function  of  yearly  flight  rate  and  flight 
experience 
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Figure  6.  A  Pilot’s  Proficiency  as  a  Function  of  Training 

For  analytical  purposes  General  Staff  analytical  group  assumes  that  the 
proficiency  of  pilots  depends  on  experience  and  annual  training  activity.  Proficiency  here 
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is  the  percentage  of  usage  the  aircraft  combat  capabilities.  Figure  (6)  shows 
proficiency  as  functions  of  flight  experience  and  the  yearly  flight  year  rate. 

D.  TASCFORM-AIR  MODEL 

The  Analytic  Scientific  Corporation  (TASC)  has  developed  a  method  to  quantify 
military  force  modernization  based  on  the  measured  performance  characteristics  of  a 
specific  military  system.  TASCFORM  provides  statistic  indicators  of  military  force 
potential  called  measures  of  effectiveness  (MOE’s).  The  measurements  of  force 
effectiveness  also  include  quantities  of  individual  weapon  systems  and  are  expressed  as 
numerical  scores.  Figure  (7)  shows  the  structure  of  the  TASCFORM  AIR  model  results. 

Individual  system  measures  of  effectiveness  for  aircraft  are  determined  by 
comparing  performance  characteristics  such  as  payload,  range,  speed,  maneuverability 
and  targeting  to  those  same  characteristics  of  a  specified  baseline  aircraft.  The  relative 
importance  of  these  characteristics  for  each  mission  is  accounted  for  through  by 
weighting  factors  developed  by  the  panel  of  experts  using  Delphi-like  techniques. 

Calculated  individual  weapon  systems  values  can  be  used  alone  or  they  can  be 
combined  with  inventory  level  and  crew  proficiency  to  produce  an  aggregate  theoretical 
force  potential.  (Regan,  p.  1-1) 

Finally  it  should  be  noted  here  that  the  TASCFORM-AIR  methodology  does  not, 
in  its  present  form,  consider  cost  as  input.  The  model  attempts  only  to  measure 
capabilities  as  an  output.  Consequently,  the  measures  of  effectiveness  are  not 
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synonymous  with  cost  effectiveness,  and  care  must  be  taken  to  avoid  reaching 
conclusions  relative  to  cost  effectiveness  when  comparing  one  system’s  measures  of 


Payload 


Range 


Maneuverability 


Speed 


FORCE  LEVEL  INDEXES 


r  " 

C3I  system  effect 

- 

Aircrew  proficiency 

i 

Logistic  and  maintenance 

- - 

’  TAC AIR  roles  distribution 
_ i _ j 


Inventory  levels 


Multi-role  capability 


Tactical  impact  of  invent] 


Figure  7.  TASCFORM  Indexes 


39 


effectiveness  with  that  of  another.  On  the  other  hand,  in  our  problem,  an  available  budget 
is  constant  for  all  alternatives.  This  budget  defines  a  number  of  aircraft  to  be  modernized 
(MIG-29  MD,  SMT),  or  increased  the  level  of  aircrew  and  groundcrew  proficiency  due  to 
better  financing.  All  these  parameters  are  included  in  force  potential  level  measures  of 
effectiveness.  Therefore,  in  force  level  we  can  consider  measures  of  effectiveness  as  a 
cost  effectiveness  index. 

A  detailed  description  of  the  methodology  is  given  in  Appendix  (A).  Excel-based 
calculations  of  alternatives  are  given  in  Appendix  (B). 

E.  LOGICAL  DECISION  FOR  WINDOWS  SOFTWARE 

Logical  Decisions  for  Windows  (LDW)  software  and  methodology  is  based  on 
Multiattribute  Utility  Theory  (Appendix  C).  It  helps  to  evaluate  decisions  quantitatively. 
Analysts  have  to  define  alternatives  and  their  variables.  LDW  methodology  describes 
alternatives  by  measures,  which  are  numerical  or  descriptive  variables  that  capture  some 
quality  of  alternatives. 

The  measures  in  LDW  are  organized  under  goals;  concerns  that  may  affect 
choice.  The  software  helps  to  organize  goals  and  measures  into  a  hierarchy.  LDW  uses 
an  overall  score,  called  utility,  to  rank  the  alternatives.  The  program  computes  an 
alternative’s  utility  by  combining  its  measure  levels  based  on  the  analyst’s  preferences. 
Hierarchy  of  goals  for  aircraft  modernization  is  shown  in  Figure  (8). 

The  LDW  provides  considerable  flexibility  in  alternative  evaluation;  therefore, 
several  basic  steps  are  common.  First,  the  analyst  must  structure  the  problem  as  a 
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Figure  8.  LDW  Modernization  Model’s  Hierarchy  of  Goals 


41 


hierarchy  of  goals,  measures  and  measure  categories.  After  that  the  analyst  has  to 
describe  the  alternatives.  The  ability  of  LDW  to  transform  the  data  to  utilities  allows  one 
to  use  a  wide  scope  of  physical,  financial  data  and  even  social  values  to  describe 
competing  alternatives. 

The  next  step  is  the  assessment  of  preferences.  LDW  provides  a  large  variety  of 
analytical  methods  to  state  relations  between  independent  and  dependent  variables. 

Finally,  the  program  ranks  the  alternatives  and  provides  the  results  in  a  number  of 
ways;  numerical,  graphical  and  probabilistic. 

The  force  level  utilities  in  our  model  are  based  on  multiplicative  approach  similar 
to  TASCFORM  methodology  and  PEREGRINE  model  (Appendix  D). 

Appendix  (F)  presents  short  description  of  different  decision  support  and 
accounting  software  available  for  decision-makers. 
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IV.  ASSESSMENT  OF  METHODOLOGIES 


As  with  any  complex  subject,  the  methodology  for  a  modernization  decision 
cannot  be  assessed  using  only  one  or  two  parameters.  Three  main  parts  of  methodology 
are  important  for  decision-makers.  The  first  part  is  the  initial  data  required  for  analysis, 
including  constants  and  variables  (input).  The  second  part  includes  algorithm, 
assumptions  and  axioms  (model).  The  third  part  of  the  analysis  is  the  results  developed 
by  the  methodology  to  make  or  support  decisions  (outcomes). 

In  addition  to  comparing  these  elements,  I  will  try  to  analyze  the  application  of 
methodologies  to  a  given  problem,  the  inherent  subjective  factors  of  both  methods  and 
the  influence  of  these  factors  on  the  model  and  on  the  results. 

A.  INITIAL  DATA 

The  initial  data  for  modernization  decision  are  numerous  aircraft  characteristics, 
such  as  performance  and  mass-geometric  data,  indexes  of  effectiveness,  economy  and 
reliability  factors,  lifetime  resources,  etc. 

One  can  distinguish  two  situations  with  aircraft  modernization  decision-making 
relevant  to  initial  data.  The  first  is  when  a  modernization  decision  is  based  on  data  of 
existing  prototype,  and  second  -  when  the  decision  is  based  on  the  estimated  data  of  the 
modernization  program.  In  the  first  case  the  decision-maker  has  more  or  less  reliable 
statistical  data  of  the  aircraft’s  physical  performance.  In  the  second  case,  an  analyst 
should  deal  with  estimates  of  data  and  performance  that  are  usually  described  with 
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certain  probabilities.  Since  the  main  task  of  this  thesis  is  methodology  assessment  and 
comparison,  I  will  assume  that  the  availability  and  the  accuracy  of  the  data  used  in  the 
thesis  computations  are  equal  to  the  data  available  for  MOD  level  of  analysis.  Available 
numerical  data  are  sufficient  for  both  TASCFORM  and  LDW  methodologies. 

There  are  several  sources  of  data  used  by  MOD  analysts  for  aircraft 
modernization  decision.  The  first  source  is  the  design  bureau  manuals  and  technical 
reports.  Most  countries  that  produce  aircraft  have  state  flight-test  centers.  Their  test 
reports  usually  have  priority  above  all  other  sources.  For  modernization  program 
selection  purposes,  in  the  case  a  prototype  doesn’t  exist,  analysts  can  use  additional 
independent  sources  such  as  scientific  research  and  publications,  specialized  journals  and 
the  estimations  of  experts.  Common  data  describing  modernization  programs  and 
status  quo  are  given  in  Figure  (5). 

B.  ADITIONAL  DATA  REQUIRED  BY  METHODOLOGIES 

Scope  of  the  data  actually  used  by  analysts  is  much  wider  than  data  given  in  the 
table.  For  example,  to  estimate  the  performance  of  an  aircraft’s  radar  one  should 
consider  several  dozens  of  parameters:  signal  power,  sensitivity,  weight,  accuracy, 
reliability,  countermeasure  sustainability,  weight,  cost,  maintainability  etc.  With  more 
details,  one  can  assess  numerous  characteristics  for  each  element.  For  instance  detection 
range  of  the  radar  can  be  described  for  high,  middle  and  low  altitudes,  for  different 
weather  conditions  and  for  various  landscapes  under  the  aircraft.  In  addition,  target  speed 
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and  relative  target  position  can  influence  detection  range.  This  shows  that  detailing  of  the 
model  may  significantly  complicate  any  problem. 

Therefore  one  of  the  most  important  problems  for  decision-makers  is  to  agree  on 
is  “How  much  initial  data  is  enough?”  To  make  such  a  decision  the  analyst  has  to  make 
several  assumptions  and  approximations,  which  help  to  simplify  the  problem.  So,  what 
are  the  main  assumptions  and  additional  data  required  by  the  TASC  and  LDW  models? 

1.  TASCFORM  Air  Model 

The  TASCFORM  Air  Model  generates  measures  of  effectiveness  for  individual 
aircraft  and  for  tactical  Air  Forces.  Appendix  (A). 

There  are  four  types  of  data  required  by  this  model: 

•  Performance  numerical  data; 

•  Expert’s  weighting  factors; 

•  Model  constants; 

•  Data  of  the  aircraft  that  is  used  for  normalization. 

Performance  in  this  model  is  actually  the  median  of  various  data  received  and 
estimated  by  design  bureaus  and  by  the  testing  centers.  All  this  data  can  be  described  by 
the  distribution  functions. 

A  weapon  potential  score  is  computed  by  scoring  the  airframe,  power  plant  and 
payload  characteristics  and  normalizing  that  score  against  a  baseline  aircraft,  the  U.S. 
F-4B.  Subjective  weighting  factors  (Figure  9)  assign  relative  importance  to  the 
characteristics  depending  on  how  they  contribute  to  the  air  combat  or  surface  attack  role. 
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In  cases  where  a  new  model  of  an  existing  aircraft  possesses  a  significant  change  in 


capability  (MiG-29md/smt),  that  model  is  treated  as  a  separate  aircraft. 
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Figure  9.  TASCFORM  AIR  Model  Weighting  Factors  and  Constants 
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presents  all  the  data  used  and  the  results  of  the  TASCFORM- AIR  model  for  MiG-29 
modernization  decision. 

2.  LDW  Aircraft  Modernization  Model 

In  contrast  to  the  TASCFORM  AIR  Model,  the  LDW  method  doesn’t  propose 
specialized  algorithm  for  any  specific  problem.  It  proposes  a  method  and  universal 
software  for  most  decision-making  problems.  A  detailed  description  of  the  LDW  isn’t 
the  subject  of  this  thesis,  so  I’ll  emphasize  only  the  main  points  of  the  solution  and 
qualities  of  the  methodology  relevant  to  my  comparison  with  the  TASCFORM  AIR 
Model. 

Actually  LDW  allows  one  to  replicate  the  TASCFORM  approach  and  to  obtain 
the  same  results.  It  can  use  all  the  formulas  of  the  TASCFORM  AIR  Model  and  its 
weighting  factors.  But  it  is  not  the  most  convenient  way  to  use  LDW. 

In  the  modeling  modernization  decisions  one  can  conclude  that  there  are  only 
certain  specific  factors,  which  are  really  influenced  by  modernization.  So,  an  effective 
solution  may  not  include  all  the  factors  and  characteristics  of  the  aircraft  but  rather,  just 
the  ones  that  actually  change.  For  comparison  purposes,  Adjusted  Aircraft  Potential 
(AAP)  is  herein  selected  as  the  main  goal  for  LDW  modernization  model.  (Figure  8) 
Similar  to  the  TASCFORM  Adjusted  Weapon  System  Potential  (AWSP),  AAP  includes 
several  lower  level  goals  such  as  aircraft  potential  and  availability.  In  contrast  to 
TASCFORM,  AAP  also  includes  economy  factors  and  remaining  age  index. 
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LDW  converts  the  alternative’s  measure  levels  into  common  units  -  utility.  The 
highest  utility  is  equal  to  one,  the  lowest  is  equal  to  zero.  Intermediate  levels  are 
assigned  with  what  is  called  a  Utility  Function,  or  more  specifically  a  Single  Utility 
Function  (SUF).  The  easiest  representation  of  SUF  is  a  straight  line.  This  is  the  most 
common  case,  and  the  default.  If  straight  lines  SUFs  aren’t  appropriate,  any  kind  of 
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Figure  10.  Data  Assigned  for  LDW  Aircraft  Modernization  Model 
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Figure  1 1 .  Forms  of  Single  Utility  Functions  Used  in  LDW  Modernization  Model 

monotone  non-linear  function  can  be  applied.  Figure  (1 1). 

On  the  next  step  LDW  computes  a  goal  utility  by  combining  the  utilities  for  each 
of  its  member  in  a  weight  average.  Similar  to  the  TASC  weighting  factors  the  LDW 
model  member’s  weight  determines  how  much  it  will  influence  an  alternative’s  utility  for 
the  goal.  LDW  provides  several  methods  for  assessing  weights.  The  simplest  method 
uses  a  simple  ordering  of  the  measures’  relative  importance.  Other  methods,  such  as 
“smart”  and  “analytical  hierarchy  process”  have  analysts  enter  the  ratios  of  the 
measure’s  relative  weights.  Another  method  called  “tradeoff’  is  based  on  the  idea  that 
equally  preferred  alternatives  should  have  equal  overall  utilities. 

Finally  LDW  provides  many  ways  to  review  results.  It  can  rank  the  alternatives 
overall  goal  or  a  measure.  LDW  sorts  alternatives  by  utilities  and  displays  them  along 
with  a  bar  representing  a  relative  score. 
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C.  APPLICATION  OF  METHODOLOGIES 

The  pursuit  of  efficiency  in  the  military  requires  a  priority  to  be  given  to  those 
programs,  that  provide  the  greatest  benefit  per  unit  of  cost.  Alternative  actions  often  have 
to  be  compared  to  determine  whether  a  change  in  the  mix  of  actions  would  increase 
efficiency.  Also  the  economic  evaluation  approaches  the  cost  in  a  common  format,  yet 
they  differ  in  the  ways  they  approach  benefits.  These  differences  play  a  critical  role  in 
developing  criteria  for  efficiency. 

So,  what  aspects  of  the  models  do  influence  the  methodology  selection,  and  what 
aspects  of  the  methodologies  influence  the  selection  of  criteria? 

1.  Hypothesis 

Two  main  hypotheses  have  influenced  the  selection  of  methodologies  and  criteria 
for  a  modernization  decision.  The  first  is  that  effectiveness  of  the  aircraft  is  primarily  a 
function  of  the  aircraft  potential  and  the  pilot  proficiency.  Therefore,  investing  in  the 
aircraft’s  modernization  is  cost-effective  only  from  a  certain  point  of  crew  proficiency. 
The  second  hypothesis  is  that  the  varying  money  available  for  modernization  may  change 
the  nature  of  the  best  solution.  In  this  case  it  depends  on  the  difference  between  marginal 
benefits  of  modernization  and  training. 

These  hypotheses  firstly  led  to  including  the  status  quo  plus  training  alternative 
for  consideration.  Secondly,  the  assumed  hypothesis  predetermined  the  choice  of  the 
force  level  indexes  of  effectiveness  as  criteria.  Finally,  both  hypotheses  provide  a  better 
understanding  of  the  real  goals  of  economic  analysis  in  the  military. 
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2. 


Criteria  of  Efficiency 


Efficiency  in  military  resource  allocation  matters  whether  these  resources  are 
being  used  to  get  the  best  value  for  money  or  not.  Efficiency  is  concerned  with  the 
relation  between  resource  inputs  and  either  intermediate  outputs  (  in  our  case  aircraft 
potential)  or  final  military  outcomes  (force  potential,  readiness,  etc.).  Also  many 
evaluations  use  intermediate  outputs  as  a  measure  of  effectiveness,  this  leads  to  possibly 
suboptimal  recommendations.  Ideally,  economic  evaluation  should  focus  on  final  military 
outcomes. 

There  are  three  main  concepts  of  efficiency  in  the  theory:  technical,  productive 
and  allocative.  Technical  efficiency  addresses  the  issue  of  using  given  resources  to  the 
maximum  advantage;  productive  efficiency  of  choosing  different  combination  of 
resources  to  achieve  the  maximum  benefits  for  a  given  cost;  and  allocative  efficiency  of 
achieving  the  right  mixture  of  military  programs  to  maximize  force  potential.  Although 
productive  efficiency  implies  technical  efficiency  and  allocative  efficiency  implies 
productive  efficiency,  none  of  the  converse  implications  necessarily  hold.  Faced  with 
limited  resources,  the  concept  of  productive  efficiency  will  eliminate  as  “inefficient” 
some  technically  efficient  resource  input  combinations,  and  the  concept  of  allocative 
efficiency  will  eliminate  some  productively  efficient  resource  allocations.  (Palmer,  p.  2) 

TASCFORM-ACR.  Model  generates  measures  of  effectiveness  for  the  individual 
aircraft  and  based  on  it  -  for  the  tactical  air  forces.  As  far  as  these  indexes  are  ratio 
comparison  of  evaluated  and  basic  aircraft,  we  can  believe  the  results  of  TASCFORM 
evaluation  is  appropriate  for  cost  effectiveness  analysis.  This  cost  effectiveness  analysis 
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type  of  analysis  measures  military  benefits  in  constructed  indexes  (weapon  potential, 
force  potential).  Since  costs  and  benefits  are  measured  in  non-comparable  units,  then- 
ratios  provide  a  yardstick  with  which  to  assess  relative  (productive)  efficiency.  This 
decision  rule  does  not,  however,  enable  us  to  evaluate  the  relative  efficiency  of  actions, 
which  provide  more  benefits  at  a  greater  cost  or  less  benefits  at  a  lower  cost.  A  major 
limitation  of  cost-effectiveness  analysis  is  its  inability  to  compare  alternatives  with 
different  natural  effects.  For  example  investments  in  air  defense  cannot  be  directly 
compared  with  those  that  improve  air  forces.  Cost-effectiveness  analysis  therefore 
cannot  directly  address  allocative  efficiency. 

Logical  Decisions  for  Windows  provides  us  with  utilities  for  individual  aircraft.  In 
the  case  of  linear  approximations  for  single-utility  functions  and  with  the  absence  of 
probabilities  in  the  model  the  result  has  an  interval  scale  meaning.  Force  level  measures 
of  effectiveness  are  the  results  of  the  multiplication  of  an  aircraft  potential  by  the  pilot’s 
proficiency  and  inventory  level.  It  lets  us  consider  the  aircraft  potential  and  force 
potential  based  on  LDW  calculations  as  an  index  for  cost-utility  analysis.  Generally 
cost-utility  analysis  is  an  adaptation  of  cost  effectiveness  analysis  which  measures  the 
alternatives  effect  on  both  the  quantitative  and  qualitative  aspects  of  military  indexes 
using  a  utility  based  measure.  Like  cost  effectiveness  analysis,  relative  efficiency  is 
assessed  using  an  incremental  ratio,  here  a  cost-utility  ratio.  An  alternative  is  deemed 
productively  efficient,  relative  to  another  one,  if  it  results  in  higher  (or  equal)  benefits  at  a 
lower  cost.  The  use  of  a  single  measure  of  military  benefit  enables  diverse  alternatives  to 
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be  compared  so  cost  utility  analysis  can  address  both  productive  efficiency  and  allocative 
efficiency. 

In  cost-utility  analysis  the  optimal  decision  rule  involves  ranking  the  incremental 
cost-utility  ratios  of  different  alternatives  and  selecting  those  with  the  lowest  ratios  (best 
value)  until  the  budget  is  depleted.  The  lower  the  incremental  ratio,  the  higher  the 
priority  in  terms  of  maximizing  military  benefits  derived  from  a  given  level  of 
expenditure.  The  point  at  which  resources  are  exhausted  defines  a  maximum  price  for  a 
unit  of  effectiveness.  Eliminating  interventions  with  an  incremental  cost  above  this  price 
in  favor  of  those  with  lower  incremental  costs  would  be  considered  an  improvement  in 
allocative  efficiency.  (Palmer,  p.  3) 

3.  Adjusting  the  Methodologies  to  the  Model 

Our  situational  model  can  be  described  as  a  model  of  the  Air  Force  unit  which 
consists  of  two  brigades  armed  with  MiG-29  (90  aircraft).  Environment,  directions, 
constraints  and  culture  are  common  for  all  alternatives.  The  difference  is  in  the  “process” 
and  “structure.”  Brigades  may  increase  their  combat  potential  by  modernizing  aircraft  or 
by  intensifying  training. 

The  TASCFORM-AIR  model  doesn’t  demand  any  structural  changes  or 
adjustments  to  presented  model.  To  construct  the  Equivalent  Force  Potential  (EFP) 
measure  of  effectiveness,  the  following  assumptions  should  be  made: 

•  Structure  of  Brigade  for  each  year  shall  be  calculated  in  accordance  with 
the  scenario.  For  example,  for  the  status  quo  alternative  the  number  of  the 
aircraft  constantly  equals  90  MiG-29S.  For  SMT  and  MD  alternatives  it  is 
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a  mix  of  the  modernized  and  basic  aircraft,  which  depends  on  the  air  depot 
productivity. 

•  Concept  of  pilot’s  proficiency  in  the  model  is  based  on  assumption  that  a 
fully-trained  pilot  can  use  100%  of  the  aircraft  potential.  To  achieve  this 
result  a  pilot  needs  five  years  of  intensive  training  (120-130  flight  hours  a 
year).  A  yearly  flight  rate  of  50  hours  let  pilots  keep  theirs  proficiency 
on  the  50%  level.  For  the  five-year  period,  I  assumed  the  linear 
approximation  of  proficiency  function. 

Pl(t)  =0.5  Flight  rate  50  hours/year; 

P2(t)  =  0.09*Y  +  0.5  Flight  rate  120  hours/year. 

Y  =  {1, 2,  3, 4, 5},  year  of  program. 

Use  of  more  complicated  proficiency  functions  shouldn’t  create  significant 
difficulties. 

As  mentioned  earlier,  the  LDW  is  a  methodological  framework  for  decision 
support.  Using  LDW,  the  analysts  create  their  own  structure  and  logic  for  the  problem 
based  on  their  model  and  intentions.  The  basic  structure  of  goal  hierarchy  for  a 
modernization  decision  is  based  on  the  Grippen  concept  of  aircraft  effectiveness 
(Appendix  F),  adjusted  TASCFORM  weighting  factors  and  PEREGRINE  multiplicative 
approach  for  the  force  potential  level  of  effectiveness.  (Appendix  D) 

In  our  case,  modernization  influences  a  comparatively  small  number  of  aircraft 
performance  measures.  It  helped  to  create  a  hierarchy  with  a  relatively  small  number  of 
measures  and  measure  categories.  (Figure  12).  As  far  as  the  main  purpose  of  this  thesis 
is  the  evaluation  of  methodologies,  not  the  evaluation  of  modernization  alternatives,  I 
assigned  reasonable  utilities  and  measures  in  the  model  where  it  was  not  available  from 
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the  references.  The  concept  of  force  structure  and  pilot’s  proficiency  remains  the  same  as 
in  the  TASCFORM-Air  model. 
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Figure  12.  LDW  Modernization  Model  Goals  Hierarchy.  Overall  Weights. 
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D.  THE  RESULTS 


Methodology  selection  demands  a  special  approach  to  the  comparison  of  the 
results.  The  main  questions  usually  are,  “What  are  the  forms  of  the  result?  What  does  it 
mean?  How  is  it  sensitive  to  changes  in  inputs?  Do  both  methods  support  similar 
decisions  and  prove  initial  hypothesis?” 


1.  The  Form  and  the  Meaning  of  the  Results 


The  TASCFORM-Air  model  generates  measures  of  effectiveness  for  individual 
aircraft.  Figure  (12)  shows  the  table  of  results  for  individual  levels:  Weapon  System 
Potential  (WSP),  Adjusted  WSP  and  Designated  AWSP. 
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1988 
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Figure  13.  TASCFORM-Air  Model  Individual  Measures  of  Effectiveness 

The  TASCFORM-Air  model  compares  a  given  aircraft  with  the  basic  aircraft  (F- 
4B).  It  does  not  directly  address  the  military  utility  of  the  basic  or  evaluated  aircraft. 
The  meaning  of  the  TASCFORM  scores  is  actually  the  rate  that  one  is  willing  to 
substitute  one  aircraft  to  another  in  order  to  have  the  same  mission  efficiency.  The  score 
of  the  basic  aircraft  in  the  model  is  equal  to  10.  The  results  in  Figure  (13)  show  us  that 
individual  scores  for  MiG-29S/  MiG-29MD/  MiG-29SMT  programs  are  rated  as 
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18.1/  20.3/  28.5.  This  means,  for  example,  that  one  MiG-29MD  is  equal  to  two  F-4Bs  in 
the  interceptor  role  and  one  MiG-29SMT  is  almost  equal  to  three  F-4Bs. 

Computing  the  measures  of  effectiveness  of  air  forces  (or  any  air  force  unit)  is  a 
two  step  process.  First,  a  designated  force  potential  score  is  computed.  This  score  is  the 
sum  of  scores  of  the  individual  aircraft  performing  a  given  role.  (Timperelake,  pp..  2-3) 
Next,  an  equivalent  force  potential  measure  of  effectiveness  is  computed  by  scoring 
relative  obsolescence,  multi-role  capability,  the  effect  of  the  numbers,  C3I  capabilities, 
logistic  and  personnel  qualities. 

Air  force  level  indexes  of  effectiveness  (Designated  Force  Potential  and 
Equivalent  Force  Potential)  have  meaning  only  for  a  particular  situation.  In  accordance 
with  the  model  and  the  problem  design  it  may  be  valuable  for  resource  allocation  tasks  or 


Figure  14.  TASCFORM-AIR  Model  Force  Level  Measures  of  Effectiveness 
rational  force  structure  definition.  Graphs  of  the  force  potential  measures  for  MiG-29 
modernization  problem  are  given  in  Figure  (14). 
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As  we  can  see,  TASCFORM  AWSP  and  DFP  scores  for  the  MiG-29 
modernization  alternatives  give  preference  to  the  SMT  program,  while  the  EFP  shows 
that  the  training  program  by  the  end  of  a  five-year  period  is  20%  more  effective  than  the 
“SMT”  program.  In  our  case  it  means  that  for  the  same  money  ($  100  million),  training 
gives  20%  better  result  than  modernization. 

In  contrast  to  the  TASCFORM- AIR  model,  the  LDW  model  provides  only  one  set 
of  results.  It  is  the  utility  for  assessed  alternatives.  LDW  computes  an  alternative’s  utility 
by  combining  its  measures  level  based  on  the  analyst’s  preferences.  The  most  common 
forms  of  the  results  are  given  in  the  Figure  (15). 
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Figure  15.  LDW  Ranking  of  Alternatives 
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ADJUSTED  AIRCRAFT  POTENTIAL 


Figure  16.  Graphs  of  the  LDW  Model  Adjusted  Aircraft  Potential 

Time  can  be  incorporated  in  the  LDW  model,  with  separate  program  runs  done 
for  each  year.  Figure  (16)  shows  the  graphs  for  alternative  utilities  in  a  time  perspective. 

The  advantage  of  LDW  is  a  wide  range  of  choices  of  methods  to  provide  and  to 
analyze  the  available  results.  There  are  sixteen  different  functions  in  the  LDW  result 
mode.  It  can  be  pure  “rank  alternative”  or  “stacked  bar  ranking”  that  show  relative 
importance  of  each  goal  in  the  final  result  (Figure  14),  or  it  can  be  diagrams  or  tables 
explaining  the  different  aspects  of  the  result. 

In  contrast  to  TASCFORM,  LDW  computes  an  interval  scale.  Changes  in  inputs 
give  changes  in  utilities,  and  those  changes  can  be  compared.  For  individual  aircraft  we 
can  compare  changes  in  utilities  from  basic  level  (status  quo)  to  the  variable  cost  of 
modernization.  Cost-utility  ratio  (CUi)  for  “SMT”  and  “MD”  alternatives  can  be 
computed  as  : 

CUi  =  (  Ui  -  Uo  )/  Ci,  where; 

Ui,  Uo  -  Utilities  of  the  assessed  alternative  and  status  quo; 

Ci  -  Variable  cost  of  the  modernization  program. 
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The  force  level  indexes  in  the  LDW  model  is  based  on  the  same  data  used  for  the 
inventory  level  and  the  crew  proficiency,  as  was  the  case  in  the  TASCFORM-AIR  model. 
The  LDW  force  level  utilities  are  given  in  Figure  (17). 


LDW  FORCE  POTENTIAL 


Figure  17.  LDW  Utilities  for  the  Force  Level 

The  LDW  results  are  similar  to  the  TASCFORM  results  at  both  the  individual  and 
at  the  force  levels.  Both  models  show  that  the  training  program  is  more  effective  at  the 
force  level,  while  at  the  individual  level  the  “SMT”  modernization  program  is 
significantly  better  than  all  the  others.  Actually  these  results  prove  our  first  hypothesis: 
If  the  level  of  crew  proficiency  is  low  -  the  investment  in  aircraft  modernization  (without 
intensifying  training)  is  not  the  best. 

The  assessment  of  the  budget  influence  led  to  a  predictable  conclusion:  Every 
program  has  limited  abilities  to  increase  efficiency  by  intensifying  training.  From  a  given 
level  of  investment,  additional  training  cannot  produce  additional  efficiency.  If  the  air 
forces  have  reached  normative  training  level,  the  only  way  to  increase  the  force  potential 
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Influence  of  the  Budget  on  the  Programs'  Efficiency 
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Figure  18.  The  Influence  of  Budget  on  Program  Efficiency 
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is  to  increase  the  weapon  potential.  Figure  (18)  shows  an  equivalent  force  potential  as 
functions  of  the  budget. 

This  graph  shows  that  the  training  programs  actually  have  reached  the  maximum 
efficiency  on  the  level  of  the  initial  budget.  The  assumption  in  the  calculations  of  the 
budget  influence  is  that  at  first  money  goes  to  modernization  rather  than  to  training. 
The  next  results  show  that  doubling  the  budget  increases  the  efficiency  of  the  “MD” 
program  by  two  times.  But  employing  the  “SMT”  program  with  four  times  the  initial 
will  result  in  the  highest  efficiency  rate  possible  in  the  situation. 
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In  the  modem  high-competitive  military  environment,  the  price  to  improve 
efficiency  is  significant.  Moreover,  the  initial  choice  of  program  will  define  the 
maximum  achievable  level. 

2.  Correlation  of  the  Results 

The  correlation  of  the  results  for  both  programs  is  obvious.  First,  both  methods 
have  similar  results  at  the  individual  weapon  level.  Second,  they  have  the  same 
tendencies  in  transition  from  one  level  to  another.  Finally,  changes  of  similar  parameters 
cause  comparable  changes  in  the  results. 


Figure  19.  Comparison  of  the  LDW  and  the  TASCFORM  Individual  Indexes 
Figure  (19)  shows  the  comparison  of  the  LDW  Adjusted  Aircraft  Potential  (AAP) 
with  the  TASCFORM-AIR  model  Adjusted  Weapon  System  Potential  (AWSP). 

The  alternatives  in  both  methods  are  ranked  in  the  same  order  with  similar  ratio 
tendencies.  The  mutual  ratios  of  the  indexes  are  1.92/  1.36/  1.00  for  LDW  and  1.57/ 
1.13/  1.00  for  TASCFORM-AIR  model.  Differences  in  the  ratio  are  most  probably 


63 


caused  by  the  initial  approach  to  the  problem.  The  LDW  considers  only  the  parameters 
that  are  influenced  by  modernization,  while  TASCFORM  includes  all  the  aircraft  s 
performances.  This  makes  the  difference  in  the  TASCFORM  results  less  then  the 
differences  in  the  LDW  results. 

Both  methods  define  training  (MiG-29S  alternative)  as  the  best  alternative  in  the 
force  level  of  analysis.  The  second  and  third  preferences  are  different.  For  example  the 
TASCFORM  approach  evaluates  “SMT”  program  10%  higher  than  the  ”MD”  program, 
while  the  LDW  gives  3%  preference  to  the  “MD”  over  the  “SMT”  alternative. 

Attempts  to  analyze  cost-utility  and  cost-effectiveness  ratios  for  “SMT”  and 
“MD”  alternatives  at  the  individual  level  led  to  controversial  results.  Figure  (20)  shows 
the  results  of  the  calculations. 
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Eo  (MiG-29S) 

Eo  (F-4B) 

SMT 
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MD 
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10.3 

Figure  20.  Cost-Utility  and  Cost-Effectiveness  Ratios  of  Modernization  Programs 

When  utilities  of  status  quo  (MiG-29S)  were  used  as  the  basic  for  calculating  the 
cost-utility  ratio,  the  “MD”  program  had  a  higher  ratio  than  the  “SMT”  program  and  is 
therefore  more  preferred.  The  same  approach  in  the  cost-effectiveness  analysis  gives  the 
opposite  result.  Here  the  “SMT’  program  had  a  better  ratio.  With  basic  aircraft  changes 
to  the  F-4B,  this  ratio  once  again  favors  the  “MD”  program.  These  results  are  good 


illustrations  of  complexities  in  the  decision-making  process.  Different  indexes  and  even 
different  concepts  of  efficiency  may  lead  to  different  conclusions.  In  our  models  the 
“SMT”  program  has  the  best  indexes  at  the  individual  level  of  analysis.  The  “MD” 
program  has  a  better  cost-utility  and  cost-effectiveness  ratio.  And  finally,  the  training 
program  has  a  better  perspective  in  the  force  level. 

3.  Sensitivity  and  Reliability 

Sensitivity  analysis  is  an  attempt  to  deal  with  uncertainty.  Sensitivity  analysis  can 
be  preformed  with  respect  to  both  predictions  of  impacts  and  their  valuation  per  unit  of 
impact.  In  particular,  it  should  convey  how  sensitive  predicted  indexes  or  utilities  are  to 
changes  in  assumptions.  If  the  ordering  of  alternatives  does  not  change  when  we 
consider  the  range  of  reasonable  assumptions,  then  our  analysis  is  robust  and  we  can  have 
greater  confidence  in  its  results. 

The  results  of  a  partial  sensitivity  analysis  for  TASCFORM-AIR  model  are 
given  in  Figure  (21).  Partial  sensitivity  is  most  appropriately  applied  to  what  the  analyst 
believes  to  be  the  most  important  and  uncertain  assumptions.  For  porpoises  of 
methodology  comparison,  I’ve  conducted  sensitivity  analysis  for  both  groups  of  the 
results  in  every  method. 

Compared  to  TASCFORM,  the  LDW  has  great  built-in  possibilities  for 
sensitivity  analysis.  The  “Sensitivity  Graph”  option  let  us  see  the  effect  of  changing  the 
weights  of  a  single  measure  or  goal.  Sensitivity  tables  let  us  see  the  effects  of  the 
changes  in  weights  of  the  goals  and  measures  quickly.  (Figure  22). 
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Figure  21.  The  Results  of  the  TASCFORM  and  the  LDW  Sensitivity  Analysis 
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Figure  22.  LDW  Sensitivity  Tables 


Reliability  of  the  results  in  both  methods  depends  on  the  quality  of  the  initial  data, 
the  model  validity  and  the  quality  of  expertise.  Nevertheless,  both  methods  are  useful  for 
certain  conditions  and  within  the  constrained  data  fields.  For  example,  let  us  assume  for 
both  models  that  the  meaning  of  the  range  and  survivability  of  the  “SMT”  alternative  is 
equal  zero.  In  reality  such  an  aircraft  has  no  military  value.  But  both  of  our  models  still 
assign  positive  indexes  to  it.  (Figure  23)  In  practice  this  means  that  analysts  using  these 
methods  should  understand  the  nature  of  the  problem  and  perform  a  pre-modeling  data 
analysis.  They  need  to  exclude  alternatives  with  invalid  characteristics  or  modify  the 
model  to  make  it  reliable.  The  LDW  software  has  the  possibility  to  include  probabilities 
and  uncertainties  into  the  analysis.  In  this  case  we  can  estimate  the  reliability  of  the 
results  numerically,  but  the  results  shift  ixom  interval  to  ordinal  scales.  This  makes  them 
impossible  to  use  for  valuable  cost-utility  ratios. 

While  the  nature  of  the  problem  and  the  data  available  have  certain  levels  of 
accuracy  and  reliability,  every  cost-effectiveness  and  cost-utility  analysis  should  be 
subjected  to  tests  of  their  sensitivity  to  the  assumptions  that  they  employ. 


Figure  23.  Indexes  and  Utilities  of  Alternatives  When  Speed  and  Range  for 
“SMT”  Alternative  are  Equal  Zero 
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D.  METHODOLOGIES 


1.  Basic  Perspectives 

Both  the  TASCFORM  AIR  Model  and  the  LDW  Model  provide  the 
dimensionless  measures  of  the  relative  technological  quality  of  a  given  system.  Both 
employ  utility  functions,  using  subjective  inputs  to  assign  weighting  factors  to 
measurable  attributes.  In  formulating  the  LDW  Model,  I  assumed  that  analysts  might  use 
different  methods  to  select  the  initial  data.  It  may  be  Delphi  approach  that  depends  on 
the  written  survey  data  gleaned  from  experienced  military  users  or  from  a  simple  expert 
estimations.  TASC,  on  the  other  hand,  created  the  TASCFORM-AIR  model  internally 
and  had  the  preliminary  version  reviewed  by  tacair  community  representatives  for 
comments  on  all  subjective  aspects.  (Timerlake,  pp..  1-5) 

The  different  methodological  approaches  and  applications  of  the  two  models 
dictate  different  levels  of  details.  By  design,  TASCFORM  is  a  relatively  simple  model. 
This  limits  the  volume  and  classifications  of  the  data  to  be  used.  The  LDW  model  based 
on  the  scenario  at  hand  and  on  the  level  of  detail  included  may  use  significantly  smaller  or 
greater  data. 

2.  Outputs  and  Applications 

Understanding  of  the  TASCFORM-AIR  model  results  for  an  individual  weapon  is 
simple.  The  score  for  the  basic  aircraft  is  10,  so  if  the  assessed  aircraft  has  scored  20,  it 
means  that  this  aircraft  is  equal  to  two  basic  aircraft  for  a  certain  mission.  To  understand 
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force  level  measures  of  effectiveness,  the  user  should  know  the  scenario.  Due  to  the 
initial  data,  results  may  vary  from  hundreds  to  thousands.  Particular  results  have 
meaning  only  for  a  comparison  of  the  given  alternatives  in  certain  situation. 

The  LDW  results  are  utilities  scored  in  the  interval  from  0.0  to  1.0.  In  accordance 
with  the  scenario  and  methods  used  to  quantify  preferences,  it  has  (in  our  case)  an 
interval  scale  meaning. 

The  ability  to  generate  scores  to  measure  the  potential  of  individual  weapon 
systems  (WSP,  AWSP)  and  basic  military  organizations  (EFP),  and  to  modify  these 
scores  to  take  into  account  qualitative  factors  make  TASCFORM  useful  in  a  number  of 
ways.  First,  it  is  useful  for  doing  systems  analysis  studies  to  track  technology  growth 
and  cost  performance  trends.  Second,  it  is  useful  to  do  macro-level  force  balance 
comparison  for  strategic  and  operational  planning.  Third  it  is  useful  for  conducting  net 
assessments  of  current  and  future  military  balances  of  any  region  in  the  world. 
(Chemiavsky,  p.  4)  TASCFORM  and  LDW  results  are  intended  primarily  for  the: 

•  Static  comparison  of  the  aircraft  potential; 

•  Force  potential  input  into  probabilistic  dynamic  wargaming  models; 

•  Input  into  force  development  and  procurement  decisions; 

•  Measurements  of  relative  force  modernization,  principally  at  the  total 
force  level,  defined  as  the  differential  of  force  equipment  potential  over 
time; 

•  Estimation  of  associated  military  investments,  when  TASCFORM  outputs 
are  coupled  with  appropriate  cost  data. 

TASCFORM- AIR  Model  is  able  to  generate  tacair  modernization  rates.  (Regan, 
p.  4-1)  These  are  based  on  either  Designated  Force  Potential  (DFP)  or  Equivalent  Force 
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Potential  (EFP),  depending  upon  whether  the  analyst  chooses  to  assess  the  extent  of  the 
modernization  embodied  in  a  tacair  force  when  incorporating  synergetic  factor. 

Modernization  rates  for  the  period  are  calculated  as  follows: 

AMRc=  dDFP/dt  =  [(DFPi  -  DFPo)/DFPo]  *100% 

FMR  =  dEFP/dt  =  [(EFPi  -  EFPo)/EFPo]  *100% 

Where:  AMRc-  collective  aircraft  modernization  rate; 

FMR  -  force  modernization  rate. 

The  model  thus  provides  the  analyst  with  the  means  for  assessing  the  effect  of 
equipment  modernization. 

3.  Form  of  the  Utility  Function 

The  TASCFORM-AIR  Model  has  four  basic  additive  parameters:  payload,  range, 
maneuverability,  and  useful  speed  to  describe  aircraft  performance.  The  values  used  for 
this  are  actual  performance  figures  normalized  to  baseline  aircraft  values.  On-board 
systems  and  features  modify  these  values  multiplicatively,  and  survivability  enters  as  a 
multiplicative  modifier  of  the  entire  weapon  system  measures  of  effectiveness  (MOE) 
rather  than  as  an  additive  term.  Individual  aircraft  MOEs  are  adjusted  for  relative 
obsolescence  and  productivity  prior  to  being  multiplied  by  inventory  levels  and  summed 
to  determine  the  basic  force  MOE.  This  is  then  adjusted  by  factors  reflecting  the  impact 
of  changing  force  levels,  C3I,  and  multi-role  capability. 

The  presented  LDW  Aircraft  Modernization  Model  is  based  on  three  basic 
additive  goals:  aircraft  potential,  crew  potential,  and  economy  factor.  Each  of  these  is  an 
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additive  function,  which  incorporate  as  many  as  ten  measures  and  ten  more  measure 
categories.  All  measures  and  measure  categories  are  subjectively  measured.  The  LDW 
utilities  for  the  Adjusted  Aircraft  Potential  are  determined  by  converting  given  measures 
to  common  units  (utilities)  and  establishing  the  relative  importance  of  the  goals  and 
measures.  To  receive  an  index  (FPI)  comparable  with  the  TASCFORM  Equivalent 
Force  Potential,  one  can  multiply  the  AAP  by  the  inventory  level. 

4.  Scenario  Applicability 

The  TASCFORM  and  the  LDW  approaches  are  consistent  in  the  area  of  scenario 
applicability.  While  the  generic  TASCFORM  methodology  was  deliberately  developed 
with  no  specific  scenario  in  mind,  it  had  always  been  recognized  that  tailoring  the 
weighting  factors  to  fit  a  given  scenario  is  an  appropriate  application  of  TASCFORM. 
This  is  regarded  as  one  of  the  virtues  of  the  model.  (Chemiavsky,  p.  5)  Similarly, 
different  LDW  models  can  be  generated  using  specific  scenarios,  simply  by  adjusting 
goal  hierarchy  and  preferences  as  appropriate. 

5.  Treatment  of  Intangibles  and  Design  Obsolescence 

The  basic  LDW  methodology  may  include  consideration  of  human  factors  and 
intangibles  in  its  models.  Obsolescence  may  be  expressed  as  usual  utility  with  negative 
utility  function.  The  given  model  results  are  time-specific  only  in  the  sense  that  an 
updated  set  of  scores  reflects  the  consensus  of  the  study  group  at  the  time  of  the  update, 
and  changes  may  be  manifested  in  a  revised  subjective  weighting  schedule.  We  can 
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conclude  that  score  for  the  same  aircraft  generated  in  different  modifications  of  models 
should  not  be  compared  for  analytical  purposes. 

TASCFORM  also  contains,  albeit  in  rudimentary  form,  consideration  of  some 
intangible  aspects  of  force  modernization,  such  as  supportability  and  maintainability  of 
systems,  the  effects  of  embodying  greater  quality  in  fever  systems,  and  vice  versa.  In 
addition,  it  contains  the  effect  of  building  multipurpose  versus  specialized  systems,  the 
effect  of  aircrew  and  groundcrew  proficiency,  the  effect  of  improved  force  command, 
communication  and  control,  and  design  obsolescence.  The  fact  that  all  of  these  aspects 
change  over  time  is  accounted  for,  and  that  will  enable  the  TASCFORM  user  to  compare 
the  effects  on  individual  systems  over  time,  as  well  as  to  assess  force-level  trends  in 
modernization. 


6.  Enhanced  Assessment  Capabilities 

The  given  examples  of  methodology  use  are  limited  to  weapon  system  potential 
and  theoretical  aggregate  force  potential.  In  many  cases  of  military  related  planning  and 
analysis  these  indexes  are  not  sufficient.  Determining  real  force  potential  requires 
assessing  the  intangible  non-weapon  aspects  of  a  military  force.  The  Analytic  Scientific 
Corporation  has  developed  a  set  of  companion  models  to  measure  the  following  (Regan, 

pp..  1-2): 

•  C3I  systems; 

•  The  potential  of  available  manpower  to  operate  and  maintain  a  force’s 
weapon  systems; 


73 


•  Effects  of  logistic  organization,  support,  and  sustainability  on  overall  force 
potential; 

•  Ability  to  mobilize  and  deploy  a  force’s  assets; 

•  Force  structure  and  resource  (fiscal)  allocations; 

•  Modernization  through  technology  development. 

Compared  to  TASCFORM  methodology,  the  LDW  is  a  more  universal  tool.  It 
helps  analysts  evaluate  any  decision  quantitatively.  One  should  define  all  alternatives 
and  variables  to  describe  them.  After  that,  LDW  helps  to  formulate  preferences  about  the 
variables  and  uses  the  information  to  rank  alternatives.  So  the  main  difference  is  that 
TASCFORM  is  a  set  of  exact  algorithms  to  be  used  in  specific  areas,  while  the  LDW  is 
universal  methodology  to  build  different  algorithms.  In  both  methodologies,  the  quality 
of  results  depend  on  the  quality  of  initial  data  and  assumptions.  TASCFORM  models 
are  likely  to  be  used  widely  by  the  public,  because  all  the  expert’s  estimations  are 
included,  while  the  LDW  models  are  strongly  dependent  on  the  quality  of  “in-house” 
expertise. 

F.  LIMITATIONS 

All  analytical  models  have  limitations.  TASCFORM  and  LDW  are  static 
assessment  models  and  have  many  of  the  limitations  of  static  models.  Both  models  are 
not  predictors  of  combat  outcomes.  They  are  indicators  of  force  potential.  In  given 
forms  and  programs’  design  both  models  do  not  account  for  dynamic  interactions  and 
cannot  measure  the  synergy  between  and  among  systems  in  combat. 
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It  was  recognized  early  that  the  availability  of  data  concerning  aircraft 
performance  and  qualities  may  vary  significantly  for  different  modernization  programs. 
The  limit  of  methodological  details  in  both  models  let  to  the  development  of  relative 
indexes  of  aircraft  and  force  potential,  not  an  attempt  to  create  unnecessarily  precise 
absolute  values. 

G.  SUBJECTIVE  FACTORS 

Several  areas  within  modernization  decision  process  are  influenced  by 
organizational  and  individual  subjective  factors.  The  most  obvious  factors  are  the 
methodology  selection,  the  collection  of  the  initial  data,  the  assigning  of  weighting 
factors,  and  finally  the  subjective  political  preferences.  Therefore,  different  methods 
may  give  different  results,  so  the  methodology  selection  plays  a  certain  role  in  the 
decision  making  process.  In  the  real  world,  it  is  often  a  matter  of  competition  between 
scientific  or  research  institutions  for  resources  and  programs.  In  our  case  the  TASC 
Corporation  has  more  authority,  experience  and  contacts  within  the  MOD.  This 
organization  promotes  and  advertises  its  products  at  all  levels  of  MOD  organizations.  It 
may  indeed  influence  MOD  decisions  more  than  any  new  scientific  company  just  in  the 
market.  On  the  other  hand,  the  more  methods  are  used  for  the  assessment,  the  less  is  the 
probability  of  mistakes. 

The  selection  of  the  initial  data  also  may  influence  the  final  decision.  It  is  clear 
that  organizations  representing  modernization  programs  will  collect  and  present  selected 
data  to  show  their  product  in  a  good  light.  In  this  case,  the  strict  and  stable  TASCFORM 
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Air  Model  is  less  influenced  by  the  initial  data  than  the  LDW  models  constructed  for  a 
particular  problem.  But  data  collected  from  independent  sources  (if  available)  may 
eliminate  this  problem.  Weighting  factors  are  the  most  important  and  most  subjective 
part  of  both  the  TASC  and  the  LDW  methodologies.  The  TASCFORM  Air  Model 
includes  12  individual  weighting  factors,  while  the  presented  LDW  model  consist  of  17 
expertly  assigned  utilities,  10  category  multipliers,  and  17  SUFs.  All  these  numbers 
represent  the  expert’s  estimations  of  data,  the  relations  or  the  relative  importance  of  any 
presenting  factors. 

TASC  emphasizes  that  their  weighting  factors  are  the  result  of  a  series  of 
conferences,  seminars  and  meetings  where  a  wide  spectrum  of  military  scientists,  pilots, 
engineers  and  operational  planners  had  made  their  estimations.  These  estimations  were 
later  statistically  transformed  to  weighting  factors  present  in  the  model.  Reliability  of 
any  model  based  on  expert  judgement  is  a  function  of  the  number  and  qualities  of  experts 
participating  in  creating  the  model.  One  more  significant  factor,  which  influence  military 
decisions,  is  the  political  preferences.  This  becomes  very  important  in  case  of  selections 
between  domestic  and  foreign  armament.  Below  are  the  results  of  a  short  questionnaire 
among  the  Naval  Postgraduate  School  international  and  US  students  concerning  their 
personal  preferences  in  armament  acquisitions.  Two  questions  were  posed: 

•  Assume  that  you  are  the  member  of  a  committee  for  acquisition  source 
selection.  In  particular  situation  you  have  three  choices:  (Select  one) 

•  To  buy  domestic  weapon; 

•  To  buy  30%  more  effective  foreign  weapon  at  the  same  price; 
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•  To  postpone  acquisition  for  five  years  and  to  invest  the  money  in 
R&D  with  a  60%  probability  of  creating  an  equaled  weapon  in  the 
future. 

•  When  would  you  prefer  armament  produced  in  a  foreign  country?  (Give 
both  figures) 

•  Armament  produced  by  allied  country  is _ more  effective  than 

domestic. 

•  Armament  produced  by  a  non-allied  country  is  _ %  more 

effective  than  domestic. 


Figure  24.  Distribution  of  Responces  for  Qestion  1 

(A-Buy  domestic;  B- Buy  30%  more  effective  foreign;  C-  Invest  in  R&D.) 

The  results  of  this  questionnaire  are  given  in  Figures  (24)  (25). 

From  the  diagrams  we  can  conclude  that  weapons  that  are  produced  by  allied 
foreign  countries  should  be  20%  more  effective  in  order  to  be  equally  preferred  to  the 
domestic  ones,  and  50%  more  effective  if  produced  by  non-allied  countries.  Reliability 
of  these  results  is  not  very  high  because  of  the  small  number  of  participants,  but  the 
common  tendency  is  obvious:  political  preferences  have  a  strong  influence  on  any 
military  decisions. 


77 


14 

12 

10 

8 

6 

4 

2 


Figure  25.  Distribution  of  Responses  for  Question  2 

(Level  of  effectiveness  for  foreign  weapon  to  be  preferred  to  domestic.) 
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V.  CONCLUSIONS 


Making  a  decision  is  like  playing  Mozart  on  the  piano:  it  is  easy  to  do,  but 
hard  to  do  well.  (Brenesal,  p.  1) 

In  today’s  corporate  world  and  bureaucratic  environment,  making  a  good  decision 
in  a  timely  fashion  is  essential.  However  complex  alternatives,  uncertainties,  and  varying 
goals  and  preferences  render  this  difficult  -  and  a  decision  once  made,  can  be  nearly 
impossible  to  explain  to  people  not  involved  in  the  decision-making  process 
Methodologies,  which  are  assessed  in  this  thesis,  allow  decisions  based  on  quantitative 
analysis.  They  also  help  to  understand  and  to  explain  the  decisions  being  made.  These 
methodologies  are  intended  to  bring  clarity  to  the  evaluation  process. 

The  TASCFORM  and  the  LDW  methodologies  have  much  in  common.  Both 
employ  utility  functions,  rely  on  subjective  inputs,  and  can  address  different  scenarios. 
The  differences  stem  from  a  fundamental  divergence  in  the  approaches  to  aggregating 
various  system  attributes.  In  the  TASCFORM  approach,  the  physical  performance 
parameters  such  as  range  and  payload  are  considered  to  be  the  basic,  additive  parameters 
that  establish  the  system's  weapon  potential.  These  basic  characteristics  are  then  modified 
by  on-board  system  capabilities,  which  enlarge  or  diminish  the  significance  of  those 
physical  performance  parameters  multiplicatively.  In  the  LDW  approach,  on-board 
system  capabilities  are  part  of  the  goal  called  “combat  effectiveness.”  Their  utilities  are 
added  to  a  common  function  in  accordance  with  the  individual  SUFs  and  preferences.  In 
the  TASCFORM,  survivability  features  are  represented  by  another  multiplicative  term, 
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which  adjusts  the  overall  "productivity"  of  the  weapon  system.  In  the  LDW  approach,  it 
is  a  separate  measure  expressed  only  by  countermeasure  utilities  just  because  the  rest  of 
the  factors,  which  influence  survivability,  didn’t  change  in  the  modernization  of  the 
MiG-29. 

There  are  possible  differences  caused  by  the  strictly  non-linear  translations  of 
performance  capabilities  into  additive  component  indices  in  the  LDW  approach. 

This  does  not  imply,  of  course,  that  there  is  no  correlation  between  LDW  and 
TASCFORM.  The  rank  ordering  of  the  alternatives,  for  instance,  is  the  same  with  the 
LDW  or  the  TASCFORM.  Both  approaches  consider  virtually  the  same  set  of  weapon 
system  attributes  and  grant  them  generally  the  same  order  of  importance.  The  difference 
is  that  the  range  in  the  indexes  between  the  most  and  least  capable  aircraft  is  1.9:1  by  the 
LDW,  but  only  1.57:1  by  the  TASCFORM 

The  TASCFORM-AIR  Methodology  does  not  consider  cost  as  input,  so  the 
measures  of  effectiveness  are  not  synonymous  with  cost-effectiveness.  However,  it 
should  be  noted  that  force  level  measures  of  effectiveness  for  a  given  air  unit  are 
subsequently  multiplied  by  the  number  of  that  aircraft’s  inventory  at  a  selected  time  to 
derive  its  contribution  to  an  overall  force  measure  of  effectiveness.  Those  inventory 
levels  are  related  to  system  costs,  thus  introducing  a  second  order  cost  effect  in  the 
model. 

The  TASCFORM-AIR  model  requires  knowing  a  large  number  of  physical 
parameters.  The  weighting  factors  and  constants  for  the  main  tactic  aircraft  missions  are 
included  in  the  methodology.  It  is  a  well-developed  model  with  a  comparatively  specific 
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application  range.  It  makes  this  method  easy  to  use  by  any  specialist  in  any  level  of 
military  planning.  Because  the  model  is  framed,  the  analyst  has  to  adjust  the  situation  to 
the  model. 

The  LDW  methodology  requires  a  good  understanding  of  the  problem  at  hand  and 
a  high  level  of  expertise  in  order  to  create  structure  (the  goal’s  hierarchy)  and  to  define 
preferences.  The  absence  of  the  strict  framework  gives  wide  possibilities,  but  demands 
experience  and  accuracy.  LDW  is  a  highly  adaptive  tool  based  on  multi-attribute  utility 
analysis  that  is  good  enough  to  be  used  in  most  decision  support  systems.  This  method, 
in  contrast  to  the  TASCFORM,  may  be  adjusted  to  any  situational  model. 

The  results  produced  are  useful  in  several  ways,  but  as  indicators  rather  than  as 
"answers."  First,  the  quality/quantity  index  values  for  weapon  systems  and  forces  can  be 
used  to  analyze  the  effects  of  technology  growth  and  modernization  measures  on  the 
conventional  force  potential.  These  can  then  be  compared  to  similar  indicators  to  assess 
ranges  of  uncertainty  regarding  these  trends.  Second,  the  index  values  are  quickly 
generated  and  the  system  can  therefore  be  rapidly  reprogrammed  to  generate  alternative 
models  based  on  different  performance  assumptions.  This  ability  to  offer  comparisons  of 
the  effect  of  different  perceptions  is  useful  in  suggesting  directions  for  more  detailed 
analyses  using  more  complex  simulation  models  and  techniques. 

It  is  important  for  an  analyst  to  reflect  the  effects  of  product  improvements,  given 
the  extent  to  which  modifications  such  as  electronic  countermeasures,  improved 
munitions  and  new  navigation  systems  have  been  added  to  existing  weapon  systems. 
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TASCFORM  caters  to  this  and  can  depreciate  quality  indexes  with  time  in  the  absence  of 
improvements  to  reflect  the  relative  decline  in  potential  of  older  weapon  systems. 

The  form  of  the  TASCFORM  results  is  more  convenient  and  understandable  for 
the  user.  It  has  the  ratio  meaning  and  can  be  easily  used  to  compose  different  indexes. 
The  understanding  of  the  LDW  resulting  utilities  is  more  complex.  In  accordance  with 
the  applied  forms  for  the  single  utility  functions  and  applied  methodology  of  preferences, 
utilities  may  have  either  an  interval  or  an  ordinal  meaning.  This  fact  makes  the  LDW 
utilities  difficult  to  use  in  the  cost-utility  ratios. 

We  have  to  mention  here  that  both  methods  support  the  initial  hypothesis.  In 
situations  with  insufficient  pilot  proficiency,  relatively  small  investments  should  be  spent 
on  training,  while  a  large  investments  should  be  directed  to  training  combined  with  the 
best  modernization  program  available. 

Sensitivity  analysis  shows  that  LDW  methodology  is  less  sensitive  to  the  partial 
changes  in  inputs  than  the  TASCFORM. 

The  areas  of  application  for  both  methodologies  are  approximately  the  same.  The 
main  difference  lies  in  the  level  of  expertise  required  by  the  methods. 

Static  quality/quantity  methodologies  do  have  some  limitations.  For  example 
TASCFORM  has  been  designed  for  simplicity  and  ease  of  use.  Complex  processes  and 
interrelationships  have  been  represented  by  simplified  characterizations  of  typical 
outcomes.  The  reliance  on  subjective  inputs  makes  both  methodologies  only  as  good  as 
the  judgements  that  have  been  incorporated.  Care  has  been  taken  to  ensure  the  quality 
and  objectivity  of  these  inputs.  Most  importantly,  static  force  assessment  methodologies 
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are  not  useful  in  predicting  combat  outcomes.  Their  utility  lies  in  shedding  light  on  the 
correlation  of  military  forces  through  side-by-side  comparison  of  conventional  force 
elements  and  associated  trends. 
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VI.  RECOMMENDATIONS 


The  real  modernization  decision-making  will  take  place  in  a  dynamic  context 
political  environment,  developing  of  the  force  structure  and  the  ongoing  military  force 
modernization  of  in  all  of  Ukraine’s  neighboring  countries.  This  will  demand  careful 
analysis.  A  variety  of  flexible  analytical  tools  must  be  used  whose  results  must  be 
integrated.  These  tools  range  from  simple  static  analyses  ("the  bean  count"),  through 
more  sophisticated  force  quantity/quality  indexes,  to  the  full  range  of  dynamic 
techniques,  such  as  simulations  and  war  games.  However,  several  cautions  are  in  order. 
(Vogt,  p.  1) 

First,  no  single  technique  or  small  set  of  techniques  can  provide  the  "correct" 
answer;  each  has  its  own  advantages  and  drawbacks.  An  understanding  of  air  force 
capabilities  requires  the  integration  of  outputs  from  a  number  of  indicators,  as  in 
economic  analysis.  Economic  analysis  is  a  complex  field  that  has  produced  a  broad  range 
of  static  and  dynamic  analytical  and  predictive  techniques.  Different  theoretical 
perceptions  of  the  overall  process  have  resulted  and  a  substantial  degree  of  uncertainty 
remains.  The  same  is  true  of  net  assessment  of  any  air  forces. 

Second,  any  comprehensive  analysis  must  acknowledge  that  the  participants  will 
disagree  about  the  main  features  of  the  air  forces  modernization  program,  due  to  the 
political  concerns  and  attempts  to  obtain  an  advantage  by  exaggerating  the  potential 
problems  of  the  particular  service.  Thus,  the  same  analyses  need  to  be  applied  to 
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different  sets  of  data,  and  the  results  examined  to  see  whether  the  disagreements  have  a 
significant  effect  on  the  overall  assessment  of  the  alternatives.  (Vogt,  p.  2) 

Cost  containment  is  not  always  synonymous  with  improved  military  capability. 
In  conditions  of  insufficient  financing  achieving  the  highest  possible  benefit  from  the 
investment  is  critical  for  military  programs.  Quantification  of  the  value  of  the  military 
benefits  of  different  modernization  programs,  is  seen  as  an  increasingly  important  process 
and  is  central  to  the  role  of  the  MOD  Armament  Department. 

Providing  advice  on  cost-effectiveness  and  cost-utility  analysis  in  the  MOD 
would  involve  the  interpretation  of  an  economic  evaluation  conducted  from  the 
perspective  of  the  military  benefits  (readiness,  sustainability,  etc)  provided  by  the 
modernization  program.  Budgetary  limits  make  economic  evaluations  more  useful  and 
acceptable,  as  often  a  new  weapon  may  have  higher  costs  while  reducing  expenditure  in 
other  sectors  of  defense  by  reducing  personnel  or  maintenance  cost.  The  aim  of 
economic  evaluation  is  to  determine  how  available  resources  can  be  used  most 
efficiently.  All  economic  evaluations  must  satisfy  two  criteria  to  qualify  as  full 
economic  evaluations.  There  must  be: 

•  Comparative  analysis  of  two  or  more  alternatives; 

•  A  full  exploration  in  all  cases  of  both  the  costs  (and  inputs)  and  the 
consequences  (or  outcomes). 

•  All  economic  analyses  should  comprise  the  following  three  dimensions: 

•  The  type  of  analysis; 

•  The  perspective  of  the  analysis; 
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•  The  type  of  costs  that  are  included. 

Types  of  analysis.  There  are  three  main  types  of  foil  economic  analysis:  cost- 
effectiveness  analysis,  cost-benefit  analysis  and  cost-utility  analysis.  All  show  the 
economic  efficiency  associated  with  a  particular  course  of  action  compared  to  an 
alternative  course  of  action.  All  these  methods  measure  costs  in  monetary  terms,  but  they 
differ  in  how  outcomes  are  expressed. 

The  perspective  of  an  analysis  is  an  important  concept  and  one  that  must  be  stated 
at  the  outset  of  a  study.  The  perspective  defines  the  range  of  costs  and  outcomes  that 
must  be  included  in  the  analysis.  If  it  is  not  stated  it  is  impossible  to  tell  whether  all 
relevant  costs  and  consequences  have  been  discussed. 

Practical  hints.  Always  look  very  carefully  at  the  literature  and  ask  the 
following  basic  questions  when  you  see  a  claim  that  a  military  program  is  “cost- 
effective”: 

•  Has  an  economic  evaluation  been  earned  out? 

•  Was  there  a  comparison  of  the  costs  and  outcomes  of  the  military 
program? 

•  Was  there  a  comparison  of  at  least  two  or  more  appropriate  alternatives? 

•  What  type  of  analysis  was  used? 

•  Was  it  appropriate? 

Finally,  each  individual  must  come  to  his  own  conclusions  about  how  much 
credibility  the  results  have  and  how  strong  the  arguments  are  that  have  been  made  in  the 
analysis. 


87 


THIS  PAGE  INTENTIONALLLY  LEFT  BLANK 


88 


APPENDIX  A.  TASCFORM-AIR  MODEL  METHODOLOGICAL  OVERVIEW 


The  TASCFORM-AIR  model  generates  measures  of  effectiveness  individual 
aircraft  and  for  tactical  air  forces.  The  model  includes  all  principal  combat  aircraft 
whose  primary  function  is  to  deliver  conventional  ordinance  against  air  and  surface 
targets.  It  does  not  include  reconnaissance,  ECM,  FAC,  and  AEW  aircraft.  Those 
aircraft  are  included  in  the  TASCFORM-C3I  model.  The  model  uses  two  broad  roles  for 
analysis  purposes:  air  combat  and  face  attack.  Within  each  role  are  a  number  of 
functions  or  missions,  as  shown  in  Table  1. 


AIR  COMBAT 

SURFACE  ATTACK 

Fighter 

Interceptor 

Close  air  support  Heavy  Bomber  battlefield  air 
Interdiction  (CAS/BAI)  launcher  (ASM) 

Interdiction  Antiship  cruise  Attack  helicopter 
missile  (ASCM)  launcher 

Heavy  bomber 

Heavy  air-to-surface  missile  (ASM)  launcher 

Table  1.  T  AC  AIR  Roles 

Computing  the  measure  of  effectiveness  for  a  single  aircraft  is  a  three-step 
process.  First,  a  weapon  potential  score  is  computed  by  scoring  the  airframe,  power 
plant,  and  payload  characteristics  and  normalizing  that  score  against  a  baseline  aircraft, 
the  U.S.  FEB.  Subjective  weighting  factors  assign  relative  importance  to  the 
characteristics  depending  on  how  they  contribute  to  the  air  combat  or  surface  attack 
role.  In  cases  where  a  new  model  of  an  existing  aircraft  possesses  a  significant  change 
in  capability,  that  model  is  treated  as  a  separate  aircraft.  Next  a  weapon  system  potential 
score  is  computed  by  scoring  on  board  systems,  such  as  weapons,  navigation,  and 
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avionics.  Finally,  an  adjusted  weapon  system  potential  score  is  computed  which  scores 
relative  productivity  and  the  existence  of  an  adaptive  enemy.  The  adjusted  weapons 
system  score  can  be  modified  by  a  relative  obsolescence  factor  to  generate  a  depreciated 
score  if  desired. 

Computing  the  measure  of  effectiveness  of  air  forces  is  a  two-step  process.  First, 
a  designated  force  potential  score  is  computed.  This  score  is  the  sum  of  scores  of  the 
individual  aircraft  performing  a  given  role.  Next,  an  equivalent  force  potential  measure 
of  effectiveness  is  computed  by  scoring  relative  obsolescence,  multirole  capability,  and 
the  effect  of  numbers,  C3I  capabilities,  logistics,  and  personnel  quality.  The  method  of 
computing  measures  of  effectiveness  for  individual  aircraft  and  air  forces  is  summarized 
bellow: 

WEAPON  POTENTIAL  (WP)  -  Basic  system  measure  of  effectiveness 
reflecting: 

•  Payload 

•  Aircraft  range,  basing  modes,  and  standoff  weapon  range 

•  Maneuverability 

•  Speed 

WEAPON  SYSTEM  POTENTIAL  (WSP)  -  Adjusts  WP  for: 

•  Target  acquisition  and  guidance/fire  control 

•  Susceptibility  to  countermeasures 

•  Weapon  enhancements 

•  Navigation 
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•  Survivability 

ADJUSTED  WEAPON  SYSTEM  POTENTIAL  (AWSP)  -  Includes: 

•  Obsolescence* 

•  Productivity 

DESIGNATED  FORCE  POTENTIAL  (DFP)  --  Basic  force  level  measure  of 
effectiveness  considers: 

•  Distribution  of  assets  to  TACAIR  roles 

•  Inventory  levels 

EQUIVALENT  FORCE  POTENTIAL  (EFP)  -  Adjusts  for: 

•  C3I  system  effects 

•  A  aircrew  proficiency 

•  Logistics  and  maintenance 

•  Multi-role  capability 

•  Tactical  impact  of  inventory  changes 

*Can  be  included  to  produce  depreciated  measures  of  effectiveness  or  excluded  at  the 
analyst's  discretion 
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APPENDIX  B.  TASCFORM  MIG-29 


SYSTEM 

POTENTIAL  LEVEL 


IFIGHTER  AIRCRAFT  MISSION 

LAST  UPDATE: 

3/4/00  IMPAC 

SYSTEM 

CATEG 

IOC 

MiG  -29S 

FGHT 

1988 

MiG  -29"MD" 

FGHT 

1988 

MiG  -29SMT 

FGHT 

1988 

1  INTERCEPTOR  AIRCRAFT 

MISSION 

LAST  T7PDATE: 

3/4/00  IMPAC 

SYSTEM 

CATEG 

IOC 

MiG  -29S 

INTP 

1988 

MiG  -29"MD" 

INTP 

1988 

MiG  -29SMT 

INTP 

1988 

PLNORM 

RNORM  MNORM 

VNORM 

8 

1800  122 

1390 

WSP@ 

IOC 

WSP00 

AWSPOO 

DAWSP0 

0 

20.3 

20.0 

20.0 

16.8 

20.4 

20.1 

20.1 

16.8 

24.5 

24.2 

24.2 

20.2 

WSP® 

IOC 

WSP  0  0 

AWSPOO 

DAWSP00 

18.7 

WBSSE& 

18.5 

21.9 

mbbsm 

liggjgjjgg 

FPL 

FR 

FM 

3 

2 

3 

DSGN00 

PL 

#  AIR  TO 

AIRSTA 

Rng 

km 

Msl  Rng 

LRAAM 

>  100km 

BSFACT 

USEFUL  T(kg- 
force) 

AIRSPD  ENTER 
TOTAL 

V  (kph)  T  * 

101.9716 

MAX 

T/OWT 

W(kg) 

0.50 

6 

1500 

0 

0 

16601.0 

18,000 

0.50 

6 

1500 

0 

0 

2,440 

16601.0 

18,500 

0.50 

8 

2100 

110 

0 

2,440 

18300.0 

19,150 

PLNORM 

RNORM 

MNORM 

VNORM 

FPL 

FR 

FM 

8 

1800 

92 

1390 

4 

3 

1 

PL 

Rng 

Msl  Rng 

USEFUL 

T  (kg- 

MAX 

force) 

#  AIR 

km 

LRAAM 

AIRSPD 

ENTER 

T/O  WT 

TO 

TOTAL 

DSGN00 

AIR  STA 

>  100km 

BSFACT 

V  (kph) 

T(kN)  * 

w  (kg) 

101.971 

6 

0.50 

6 

1500 

0 

0 

2445.0 

16601.0 

18000 

0.50 

6 

1500 

0 

0 

2445.0 

16601.0 

18500 

0.50 

8 

2100 

110 

0 

2445.0 

18300.0 

19150 
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FV 

2 

STOLSEAPLA  CARRIE 
NE  R 

450  450  750 

VSTOL 

200 

TFNGM 

0.2 

TFGM 

0.8 

TADAY 

1.0 

TANITE 

1.0 

TA1-00  GME1-00  CMGM1- 

00 

Nl-00 

TA2-00 

GM2-00  CMGM2- 
00 

N2-00  TANGM0 

0 

2.0 

1.6 

1.0 

4 

2.0 

1.2 

0.9 

2 

1.0 

2.0 

1.6 

1.0 

4 

2.0 

1.2 

1.0 

2 

1.0 

2.0 

1.6 

1.1 

4 

2.0 

1.2 

1.0 

4 

1.0 

FV 

STOLSEAPLAN 

CARRIER 

VSTOL 

TFNGM 

TFGM 

TADAY 

TANITE 

E 

2 

450 

450 

750 

200 

0.1 

0.9 

1.0 

1.2 

TA1-00 

GMEF1- 

CMGM1- 

Nl-00  TA2-00 

GM2-00 

CMGM2- 

N2-00 

TANGM00 

00 

00 

00 

2.0 

1.6 

1.0 

2  1.2 

1.4 

0.9 

4 

1.0 

2.0 

1.6 

1.1 

2  1.2 

1.4 

1.0 

4 

1.0 

TALWX 

TAAWX  GMVSE 

GMVHO  GMVMU 

GMVOBS  GMBSE 

GMBHO 

GMBMU 

MI 

ME  LT 

MI 

ME 

LT 

1.2 

2.0 

0.8 

1.0  1.2 

1.4 

0.8 

1.2 

1.6 

SURVIVABILITY  FACTORS,  AIR 

COMBAT 

ENTER  VALUES  FROM 

CG3 

TABLE 

NAV 

PS/RC  VELOCI 

SIZE 

SMOKE 

ACTCM 

PASSCM 

1  S 

TY 

1 

1.2 

1.0 

35  25 

15 

10 

0 

5 

1 

1.2 

1.0 

35  25 

15 

10 

0 

5 

1 

1.2 

1.0 

35  25 

15 

10 

0 

5 

TALWX 

TAAWX 

GMVSEMI 

GMVHOME  GMVMULT 

GMVOBS 

GMBSEMI 

GMBHOME 

GMBMULT 

1.6 

2.0 

0.8 

1.0  1.2 

1.4 

1.0 

1.6 

2.0 

NUMBER 

ENTER 

SURVIVABILITY  FACTORS, 

CR 

AIR  COMBAT 

OF  N/ GUIDED 

CE3  TO 

ENTER  VALUES  FROM  TABLE 

CG3 

SYSTEMS 

WE00 

NAV 

PS/RC  VELOCIT 
Y 

SIZE 

SMOKE 

ACTCM 

PASSCM 

i 

1.2 

1.0 

10  25 

15 

10 

5 

10 

i 

1.2 

1.0 

10  25 

15 

10 

10 

10 

i 

1.2 

1.0 

10  25 

15 

10 

15 

15 

94 


GMBGM 

CMVHI 

CMffl 

CMAVG 

CMLO 

CMVLO  NAVPOO  NAVGOO 

NAVEX 

UL 

R 

D 

2.0 

0.7 

0.8 

0.9 

1.0 

1.1 

0.8 

1.0 

1.0 

ENTER 

ALGORY 

CR 

CI3TO 

CK3 

THMS 

FIRE/FO 

STANDO 

HARDEN 

REDUN 

SORTIE 

A00 

PLTERM 

RG 

FF 

10 

10 

10 

5 

1.0 

1 

2.3 

10 

10 

10 

5 

1.0 

1 

2.3 

10 

10 

10 

5 

1.0 

1 

3.0 

GMBGMUL 

CMVHI 

CMHI 

CMAVG 

CMLO 

CMVLO 

NAVPOOR 

NAVGOOD 

NAVEX 

3.0 

0.7 

0.8 

0.9 

1.0 

1.1 

0.8 

1.0 

1.0 

ENTER 

ALGORYT 

CR 

Cl  3  TO 

CK3 

HMS 

FIRE/FO 

STANDOF 

HARDEN 

REDUN 

SORTIE 

A00 

PLTERM 

RG 

F 

15 

20 

10 

5 

1.0 

i 

2.3 

15 

20 

10 

5 

1.0 

i 

2.5 

20 

20 

10 

5 

1.0 

i 

3.0 

DRAG 

SRLO 

SRAVG 

SRHI 

K 

Uls 

Ulmd 

ULsmt 

0.8925 

0.8 

1.0 

1.2 

2.0 

20.0 

25 

30 

ALGORITHMS 

RTERM 

Ps 

MTERM 

VTERM 

PU00 

RAWSIG 

SURV 

AP 

MA 

1.7 

241 

5.9 

3.5 

3.0 

125.0 

1.1 

13.4 

1.7 

235 

5.8 

3.5 

3.1 

125.0 

1.1 

13.2 

2.6 

250 

6.1 

3.5 

3.1 

125.0 

1.1 

15.2 

DRAG 

SRLO 

SRAVG 

SRHI 

K 

UL 

0.9192 

0.8 

1.0 

1.2 

2.0 

25.0 

ALGORYTHMS 

RTERM 

Ps 

MTERM 

VTERM 

PU00 

RAWSIGM 

SURV 

AP 

A 

1.7 

242 

6.0 

3.5 

2.2 

125.0 

1.1 

13.4 

2 . 0 

235 

6.1 

3.5 

2.4 

130.0 

1.2 

14.1 

2.6 

251 

6.2 

3.5 

2.7 

145.0 

1.4 

15.3 
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EQIVALENT  FORCE 

POTENTIAL 

YEARS 

MiG  -29S 

1310.629  1276.135  1241.008  1205.206  1168.679 

MiG  -29"MD" 

1366.746  1392.761  1422.805  1394.156  1365.109 

MiG  -29SMT 

1388.216  1432.409  1477.181  1443.263  1408.756 

MEAN 

AWSPS 

19.05382 

20.2 

26.33201 


DESIGNATED  FORCE 

POTENTIAL 

YEARS 

mmmm 

||ii§ 

773.2712  867.772  955.5764  1036.477  1110.245 

90 

90 

90 

781.7789  716.9934  813.8443  797.4572  780.8424 

30.0 

60.0 

90 

919.5543  948.8277  978.4847  956.0174  933.1601 

10 

20 

30 

(COEFFICIENTS 

Kc3i  Kmrc  Logisti  Mainten  Ground 
c  ance  crew 


1 

1.3 


1 

.0 


Air  Cre^ 
Prof icie 

ncy 

1 S#I5MP 

0.59  0.68  0.77  0.86  0.95 

0.5  0.5  0.5  0.5  0.5 

0.5  0.5  0.5  0.5  0.5 

Obsolet 

ness 

0.764285 

0.815009 

0.847684 

0.74417 

0.799631 

0.835218 

0.723686 

0.784053 

0.822626 

0.702808 

0.768265 

0.809905 

0.681508 

0.752259 

0.797049 

1 

1.1 

1.2 


1 

0.8 

0.8 
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WEAPON  SYSTEM  LEVEL  INDEXES 
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APPENDIX  C.  MULTI-ATTRIBUTE  UTILITY  THEORY 

Multi -Attribute  Utility  Theory  (MAUT)  applies  to  situation  with  uncertainty  and 
multiple,  often  conflicting,  objectives.  For  situation  where  the  decision  variables  are 
independent,  a  weighted  addition  of  utilities  (i.e.,  linear)  is  used  to  produce  an  ordinal 
ranking  of  alternatives.  When  interactions  between  the  variables  exist,  multiplicative 
terms  are  introduced,  resulting  in  a  multi-linear  overall  utility  function.  A  general  form 
of  this  equation  is: 

n 

u(x)  =  Yjki  + 

1=1 

n  n 

+  X  'Z'Zkin  uSx)  Uj(x?UxXi  +  •••  + 

t-1  j>  1  /= 1  j>i  e>j 

k\23....N  Hi  ^x)U2^Xo)—uSx) 

where: 

1.  n  is  normalized  by  u  ,  ...  =  0  (the  least  preferred  level  of 

*  *  * 

all  measures)  and  u  (xi>  X-> >  •••  X?)  ( the  most  preferred  level  of  all  measures). 


2.  Ui^Xj )  a  conditional  utility  function  of  j£.  normalized  by  ^.(j \\)=  0 

and  Uj(Xi)  =  L 

3.  The  scaling  constants  can  be  evaluated  by:  km  n~^~ 


Yu  (x°X,)  +  -  +  (-l)  Yu(x*i  ’Xj’X°P  +  (- 1)  Yu(xi 


i,;>i 
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APPENDIX  D.  PEREGRINE  MODEL 


The  basic  PEREGRINE  model  is  multiplicative.  That  is,  if  "A"  is  the  availability 
term,  "M"  the  mission  effectiveness  term,  and  "S"  the  survivability  term,  the  overall 
system  potential  P=A*M*S.  Consider  two  hypothetical  aircraft.  One  has  outstanding 
availability  and  mission  effectiveness  scores  but  has  terrible  survivability  while  the  other 
is  "good  but  not  great"  in  all  three  areas.  Suppose  A,  M,  and  S  are  scaled  from  1  to  10. 
Our  first  aircraft  might  rate  9-9-1  while  our  second  rates  6-6-6.  An  additive  model  would 
score  the  two  aircraft  as  19  versus  18,  indicating  that  the  first  aircraft  by  virtue  of  its  very 
high  availability  and  mission  effectiveness  is  slightly  superior  to  the  "good  but  not  great" 
aircraft.  This  despite  the  fact  that  it  is  going  to  get  shot  out  of  the  sky  any  time  it  faces  a 
capable  air  defense  system.  The  multiplicative  model  would  score  the  two  planes  as  81 
for  the  first  versus  216  for  the  second,  indicating  that  the  second  aircraft  is  much  better 
since  it  has  no  crippling  weaknesses.  This  is  far  more  supportable  on  both  intuitive 
grounds  and  by  historical  examples. 

The  "A",  "M",  and  "S"  factors  each  are  composed  of  sub-terms  and  these  may 
combine  additively,  multiplicatively,  or  in  some  other  fashion  depending  on  the 
relationship  between  the  factors  involved.  For  example,  the  ability  of  one  aircraft  to  shoot 
another  down  will  depend  on  (among  other  factors)  the  first  aircraft’s  ability  to  detect  the 
target,  to  maneuver  into  shooting  position,  and  to  hit  the  target  with  its  weapons.  This  is 
best  expressed  by  a  multiplicative  relation  between  these  factors  since  an  inability  to  do 
any  one  of  them  will  prevent  the  aircraft  from  hitting  the  target.  But  the  effectiveness  of 
the  aircraft’s  radar  guided  missiles  (for  use  at  long  range)  is  independent  of  the 
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effectiveness  of  its  infrared  missiles  (for  use  in  a  short  range  "dog  fight").  Even  if  it  had 
zero  capability  to  use  one  of  these  weapons  types,  it  might  have  great  effectiveness  with 
the  other.  Thus  multiplying  a  short-range  missile  factor  by  a  long-range  factor  to  compute 
an  overall  value  for  the  aircraft’s  weapons  suite  would  be  inaccurate.  Some  other  method 
of  combining  the  factors  would  be  more  realistic. 

Though  the  general  purpose  of  PEREGRINE  is  to  provide  an  overall  measure  of 
system  potential  for  fighter  aircraft  in  different  missions,  that  potential  will  be  not  just 
mission  but  scenario  dependent.  For  example,  if  two  aircraft  are  being  compared  in  the 
same  mission  and  one  has  adequate  survivability  and  very  high  lethality  while  the  other 
reverses  those  characteristics,  the  relative  value  of  the  two  will  depend  greatly  on  enemy 
defenses.  But  even  the  "adequate"  survivability  might  suffice  if  the  aircraft  is  supported 
by  advanced  fighters  and  specialized  defense  suppression  aircraft.  No  static  model  can 
capture  all  these  dynamics  but  PEREGRINE  attempts  to  approximate  them. 
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APPENDIX  E.  LDW  RESULTS  IN  EXCEL 


INDIVIDUAL  WEAPON  SYSTEM  POTENTIAL _ 

ADGUSTED  AIRCRAFT  POTENTIAL  I  UTILITIE 

_  S 


LDW  FORCE  POTENTIAL  FP  =  N1  *AAP1  +  N2  *  AAP2 


INVENTORY  LEVEL 


PILOT’S  PROFICIENCY 


INVENTORY  LEVEL 


PILOT’S  PROFICIENCY 


TYPE 

2000 

2001 

2002 

2003 

2004 

2005 

MiG  -29S 

0.5 

0.59 

0.68 

0.77 

0.86 

0.95 

MiG  -29MD 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

MiG-29SMT 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 
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UTILITIES 


LDW  UTILITIES  FOR  THE 
AIRCCRAFT 


LDW  FORCE  POTENTIAL 


2000  2002  2004 

YEARS 
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APPENDIX  F.  GRIPPEN  CONCEPT  OF  AIRCRAFT  EFFICIENCY 
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APPENDIX  G.  DECISION-SUPPORT  AND  ACCOUNTING  SOFTWARE 

EVALUATION  TOOLS 

EXPERT  CHOICE.  Expert  Choice,  Inc.  offers  two  main  products  for  sale, 
ECPro  (for  use  by  individual  decision  makers)  and  TeamEC  (a  group  decision-making 
tool).  Both  products  are  for  Windows  and  are  based  on  the  analytic  hierarchy  process 
(AHP).  The  AHP  is  a  decision  tool  that  is  for  use  with  decisions  involving  multiple 
attributes  or  criteria  and  multiple  alternative  solutions,  www.expertchoice.com 

LOGICAL  DECISIONS  FOR  WINDOWS.  Logical  Decisions  for  Windows 
(LDW)  is  a  decision-support  software  tool  that  uses  both  the  AHP  and  other  multiple 
variable  decision  approaches  to  help  decision  makers  make  choices  in  complex  problem 
situations.  Like  Expert  Choice,  LDW  evaluates  multiple  attributes  and  multiple 
alternatives.  The  software  can  be  tailored  to  a  variety  of  decision  contexts,  such  as  the 
accounting  software  selection  decision,  www.logicaldecisions.com 

REQUIREMENTS  ANALYST.  Computer  Training  Services,  Inc.,  has  sold 
Requirements  Analyst  for  many  years.  The  software  is  regularly  updated  with  new  data 
about  accounting  software  products.  Decision  makers  supply  information  about  the 
software  features  they  either  need  or  would  like  to  have,  and  Requirements  Analyst 
produces  a  list  of  the  features  associated  with  the  optimal  accounting  software  choice. 
ww.ctsguides.com 

SOFTWARE  COMPARE.  The  home  page  address  for  Practitioners  Publishing 
Company,  vendors  for  this  product,  is  www.ppcinfo.com.  Practitioners  Publishing 
Company  also  sells  a  manual,  Guide  to  Installing  Microcomputer  Accounting  Systems,  at 
this  site. 
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THE  ACCOUNTING  LIBRARY.  The  Accounting  Library  (TAL)  is  a  software 
tool  designed  specifically  for  the  accounting  software  selection  decision.  Software 
vendors  furnish  data  about  their  software  features  to  the  program’s  developer.  TAL  users 
supply  data  about  those  software  features  that  are  of  particular  importance.  The  software 
includes  more  than  100  software  programs  and  more  than  1,500  factors  or  software 
features.  There  are  several  versions  of  the  software  available  at  www.excelco.com. 

ACCOUNTING/FINANCIAL  MANAGEMENT  SOFTWARE-RELATED 
INTERNET  SITES 

www.accountingdirecdtory.com  This  site  includes  a  directory  of  accounting 
software  companies  and  other  resources  for  information  about  accounting  systems.  From 
the  site,  you  may  access  a  consulting  service,  Accountsoft,  that  chooses  your  accounting 
software  solution  for  a  fee. 

www.acctg2000.com  A  website  that  offers  assistance  in  selecting  the  best 
accounting  software  package  for  your  business.  The  site  includes  a  list  of  accounting 
software  publishers’  names  and  sources  of  software  support.  There  are  links  to  a  variety 
of  other  sites,  such  as  educational/training  sources,  that  can  also  provide  you  with  help. 

www.cpaonline.com  The  site  includes  news  items  related  to  technology  that  are 
likely  to  be  of  interest  to  accountants.  The  site  also  has  links  to  a  seminar  series  offering 
guidance  in  accounting  software  selection  (Business  and  Accounting  Software  Update),  a 
summary  of  accounting  software  packages,  and  a  list  of  accounting  software  consultants 
in  your  business  location. 

www.ctsguides.com  This  is  the  home  page  for  Computer  Training  Services,  a 
Rockville,  Md.,  company  that  sells  products  and  services  for  consultants  and  managers  to 
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use  in  selecting  accounting  software.  The  company  sells  software  guides  for  special 
industries  in  addition  to  their  premier  product.  The  Requirements  Analyst.  The  site  also 
includes  advice  about  the  software  selection  process  from  needs  analysis  to 
implementation. 

www.excelco.com  The  site  is  an  accounting  system  selector  that  advertises  The 
Accounting  Library,  You  can  buy  all  versions  of  The  Accounting  Library  at  this  site  or 
purchase  consulting  services  that  rely  on  it. 

www.fsforum.com  This  site  is  a  not-for-profit  site  where  financial  systems 
professionals  exchange  information  in  discussion  groups  related  to  their  interests.  The 
site  includes  resources  such  as  a  free  software  search  service,  vendor  links,  a  resource 
library  with  articles  about  financial  systems,  and  Y2K  resources. 

www.k2e.com/ac/accountingsoftware.htm  A  comprehensive  Web  source  with 
information  about  accounting  software  for  accountants  and  consultants.  The  site, 
developed  by  K2  Enterprises,  a  company  specializing  in  offering  accounting  system 
selection  support,  includes  information  on  accounting  software  packages,  software 
evaluations,  links  to  software  vendors,  manual  and  book  resources  available,  and 
information  on  software  selection  seminars. 

www.mamag.com/strategicfinance/vendors.htm  The  online  listing  of  software 
vendors  from  Strategic  Finance. 


www.softwarenews.net/buyers/account.htm  A  software  news  service  for 
accounting  software  buyers  containing  links  to  software  vendors. 
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